Smalltalk › Pharo › Pharo Smalltalk Developers

Basic versioning of GitHub repositories from within Pharo

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

27 messages Options

Skip Lentz-2

Basic versioning of GitHub repositories from within Pharo

Hi everyone,

As part of my internship I am creating bindings to the GitHub API in Pharo.

As a prototype and demo, I have created a small tool last week to do some basic versioning, namely checking out a version, committing a version and showing a log of commits along with a branch tree.

Here’s a screenshot: http://i.imgur.com/iMfWOvp.png

The repository of the bindings and the tool is here: http://smalltalkhub.com/#!/~Balletie/GitHub

To load the tool into your image, execute:

(ConfigurationOfGitHub project version: #development) load: #tool

Keep in mind that this is tied to GitHub, since internally it uses the API. A nice side effect of this is that everything can be done in-memory. That is, there’s no local copy on the filesystem. One does not even need git installed.

That being said, feel free to take off with my prototype and make it work with e.g. the LibGit bindings in Pharo.

Known bug:

- When selecting a different repository from the dropdown while a version is also selected in the log, one gets a DNU. To work around this for the time being, just deselect the version before you switch repositories.

Let me know what you think and feel free to ask some questions.

Skip

Max Leske

Re: Basic versioning of GitHub repositories from within Pharo

Sweet! You’re not using the LibGit bindings for this, are you?

Cheers,

Max

On 23 Nov 2015, at 16:25, Skip Lentz <[hidden email]> wrote:

Hi everyone,

As part of my internship I am creating bindings to the GitHub API in Pharo.
As a prototype and demo, I have created a small tool last week to do some basic versioning, namely checking out a version, committing a version and showing a log of commits along with a branch tree.

Here’s a screenshot: http://i.imgur.com/iMfWOvp.png

The repository of the bindings and the tool is here: http://smalltalkhub.com/#!/~Balletie/GitHub
To load the tool into your image, execute:

(ConfigurationOfGitHub project version: #development) load: #tool

Keep in mind that this is tied to GitHub, since internally it uses the API. A nice side effect of this is that everything can be done in-memory. That is, there’s no local copy on the filesystem. One does not even need git installed.

That being said, feel free to take off with my prototype and make it work with e.g. the LibGit bindings in Pharo.

Known bug:
- When selecting a different repository from the dropdown while a version is also selected in the log, one gets a DNU. To work around this for the time being, just deselect the version before you switch repositories.

Let me know what you think and feel free to ask some questions.

Skip

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

> On Nov 23, 2015, at 5:45 PM, Max Leske <[hidden email]> wrote:
>
> Sweet! You’re not using the LibGit bindings for this, are you?

No, I’m not. But it would be interesting to try something similar to this with the LibGit bindings :). I might take a stab at this..
Right now it’s tied to the GitHub API which is kind of a bummer, but for prototype purposes it’s alright in my opinion.

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

Actually its not a bummer at all, Github has an amazing set of features, 99% why people use git, including me, is because of Github. Taking also into account that no other alternative of Github is as remotely as popular and the biggest competitor , gitorious , just died.

Having Github API suppoer inside Pharo is as important if not more important than having git support. You can even commit files with the Github API, so its far from just a browser, issue and wiki tracker. I will definetly take a look at your code and I would like to thank you as a huge fan of Github, very much.

On Mon, Nov 23, 2015 at 6:51 PM Skip Lentz <[hidden email]> wrote:

> On Nov 23, 2015, at 5:45 PM, Max Leske <[hidden email]> wrote:
>
> Sweet! You’re not using the LibGit bindings for this, are you?

No, I’m not. But it would be interesting to try something similar to this with the LibGit bindings :). I might take a stab at this..
Right now it’s tied to the GitHub API which is kind of a bummer, but for prototype purposes it’s alright in my opinion.

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

> On Nov 23, 2015, at 5:54 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> Actually its not a bummer at all, Github has an amazing set of features, 99% why people use git, including me, is because of Github. Taking also into account that no other alternative of Github is as remotely as popular and the biggest competitor , gitorious , just died.
>
> Having Github API suppoer inside Pharo is as important if not more important than having git support. You can even commit files with the Github API, so its far from just a browser, issue and wiki tracker. I will definetly take a look at your code and I would like to thank you as a huge fan of Github, very much.

Thank you, I knew someone would like it. If you want to take a stab at creating a commit, you should look at the CommitBuilder class and its class comment. You can pass it a directory reference as an argument and it will commit all the files once you send #push to it. (This also works with an in-memory directory using the MemoryStore for the FileSystem, which I think is an interesting use-case).

I will expand it with some more documentation within the coming week. As for the API, there are still some TODOs :).

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

In reply to this post by kilon.alios

hmm i put my user name and password I press enter and gives me a MNU: receiver of "asZnUrl" is nil

On Mon, Nov 23, 2015 at 6:54 PM Dimitris Chloupis <[hidden email]> wrote:

Actually its not a bummer at all, Github has an amazing set of features, 99% why people use git, including me, is because of Github. Taking also into account that no other alternative of Github is as remotely as popular and the biggest competitor , gitorious , just died.

Having Github API suppoer inside Pharo is as important if not more important than having git support. You can even commit files with the Github API, so its far from just a browser, issue and wiki tracker. I will definetly take a look at your code and I would like to thank you as a huge fan of Github, very much.

On Mon, Nov 23, 2015 at 6:51 PM Skip Lentz <[hidden email]> wrote:

> On Nov 23, 2015, at 5:45 PM, Max Leske <[hidden email]> wrote:
>
> Sweet! You’re not using the LibGit bindings for this, are you?

No, I’m not. But it would be interesting to try something similar to this with the LibGit bindings :). I might take a stab at this..
Right now it’s tied to the GitHub API which is kind of a bummer, but for prototype purposes it’s alright in my opinion.

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

> On Nov 23, 2015, at 6:03 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> hmm i put my user name and password I press enter and gives me a MNU: receiver of "asZnUrl" is nil

Hang on, let me try in a fresh image. Thanks for the feedback by the way

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

And I thank you a ton, for creating this tool , I was thinking to learn Github API because I wanted to make an updater for pharo github projects, an updater that can detect when there is a new commit and fetch the latest version and alert the user about the update automagically. I am also interested into integrating github issues and even wiki inside pharo. You just made my life a lot easier.

On Mon, Nov 23, 2015 at 7:06 PM Skip Lentz <[hidden email]> wrote:

> On Nov 23, 2015, at 6:03 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> hmm i put my user name and password I press enter and gives me a MNU: receiver of "asZnUrl" is nil

Hang on, let me try in a fresh image. Thanks for the feedback by the way

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

In reply to this post by kilon.alios

> On Nov 23, 2015, at 6:03 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> hmm i put my user name and password I press enter and gives me a MNU: receiver of "asZnUrl" is nil

I debugged together with Martin and this error occurred when he didn’t fill in his password correctly. I still have to check for HTTP error codes and the like from responses.

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

nope my password is definitely correct and the user name.

On Mon, Nov 23, 2015 at 7:34 PM Skip Lentz <[hidden email]> wrote:

> On Nov 23, 2015, at 6:03 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> hmm i put my user name and password I press enter and gives me a MNU: receiver of "asZnUrl" is nil

I debugged together with Martin and this error occurred when he didn’t fill in his password correctly. I still have to check for HTTP error codes and the like from responses.

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

In reply to this post by Skip Lentz-2

Issue browsing is just the tip of the iceberg

we talking here

1) github notifications

2) staring of repositories, track all your fans who star or watch your repos

3) pull requests

4) events

5) gihub hooks , so you can trigger other actions when something happens in a repo

6) gists, a direct competitor to publishing playgrounds online

7) git data of course, which includes the usual suspect (commits, branches, pull requests etc etc)

8) yes even emojis that github can use

9) gitignore

10) organisations , essential for big teams

11) the super powerful search engine of github

12) user names, email, websites etc etc

13) wiki pages

14) github web pages

and the list goes on and on

of course putting all that inside pharo is 14 pharo projects by themselves :D

but this tool definitely opens the door to such distant future

On Mon, Nov 23, 2015 at 7:34 PM Skip Lentz <[hidden email]> wrote:

> On Nov 23, 2015, at 6:03 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> hmm i put my user name and password I press enter and gives me a MNU: receiver of "asZnUrl" is nil

I debugged together with Martin and this error occurred when he didn’t fill in his password correctly. I still have to check for HTTP error codes and the like from responses.

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

In reply to this post by kilon.alios

> On Nov 23, 2015, at 6:35 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> nope my password is definitely correct and the user name.

Hmm ok.. In that case I really don’t know.
Could you look at the stack frame to see if any of the methods are related with avatars?
The avatar url should always have a non-nil value since all users get one assigned, so it’s only nil if the response was unsuccessful (i.e.: wrong username/password).

Another thing which you can try is create a Personal Access Token on GitHub via Home > Settings > Personal Access Tokens.
This token can be filled into the username field of the tool and you can then leave the password blank.

If this doesn’t work then your error is certainly a different one than the one I debugged with Martin.

I’m logging of now, I can only get back on this on Wednesday as I’m preoccupied tomorrow..

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

ok will take a look when I have time, i tried via token, same problem so something is wrong with the code.

By the way do you develop this using the latest pharo 5 image and latest pharo vm ?

On Mon, Nov 23, 2015 at 7:51 PM Skip Lentz <[hidden email]> wrote:

> On Nov 23, 2015, at 6:35 PM, Dimitris Chloupis <[hidden email]> wrote:
>
> nope my password is definitely correct and the user name.

Hmm ok.. In that case I really don’t know.
Could you look at the stack frame to see if any of the methods are related with avatars?
The avatar url should always have a non-nil value since all users get one assigned, so it’s only nil if the response was unsuccessful (i.e.: wrong username/password).

Another thing which you can try is create a Personal Access Token on GitHub via Home > Settings > Personal Access Tokens.
This token can be filled into the username field of the tool and you can then leave the password blank.

If this doesn’t work then your error is certainly a different one than the one I debugged with Martin.

I’m logging of now, I can only get back on this on Wednesday as I’m preoccupied tomorrow..

stepharo

Re: Basic versioning of GitHub repositories from within Pharo

In reply to this post by Skip Lentz-2

I suggest all interested parties to
    - review the code of skip
    - let him some time to improve
    - help if you want
The goal of skip internship with us is to build a better code review system for Pharo.
And we decided to use existing infrastructure instead of building our own seaside-based yet another server to maintain.

Stef

Le 23/11/15 16:25, Skip Lentz a écrit :

Hi everyone,

As part of my internship I am creating bindings to the GitHub API in Pharo.

As a prototype and demo, I have created a small tool last week to do some basic versioning, namely checking out a version, committing a version and showing a log of commits along with a branch tree.

Here’s a screenshot: http://i.imgur.com/iMfWOvp.png

The repository of the bindings and the tool is here: http://smalltalkhub.com/#!/~Balletie/GitHub

To load the tool into your image, execute:

(ConfigurationOfGitHub project version: #development) load: #tool

Keep in mind that this is tied to GitHub, since internally it uses the API. A nice side effect of this is that everything can be done in-memory. That is, there’s no local copy on the filesystem. One does not even need git installed.

That being said, feel free to take off with my prototype and make it work with e.g. the LibGit bindings in Pharo.

Known bug:

- When selecting a different repository from the dropdown while a version is also selected in the log, one gets a DNU. To work around this for the time being, just deselect the version before you switch repositories.

Let me know what you think and feel free to ask some questions.

Skip

kilon.alios

Re: Basic versioning of GitHub repositories from within Pharo

I suggest all interested parties to

- review the code of skip

absolutely I already have begun to do it

- let him some time to improve

no rush at all from my side, he has as much time as he wants

- help if you want

I offer to design a custom GUI for it ;) but first I will let you get the functionality right and I can learn from the code as well in the meantime because I am not familiar with the REST API at all :)

The goal of skip internship with us is to build a better code review system for Pharo.
And we decided to use existing infrastructure instead of building our own seaside-based yet another server to maintain.

good choice, afterall pharo already greatly benefits from taking advantage of existing technologies like CI, Cairo, OpenGL, git , mercurial, MongoDB etc . Keep the great work and thanks for motivating me to use and learn Pharo, more and more.

Stef

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

In reply to this post by Skip Lentz-2

Hi again,

I wil try to respond to most of you in this e-mail:

@ Thierry:

> Would you be able to retrieve the same information from github that GitFileTree does when it extract package version history information?

It would be nice to have some less metadata in the repository (or none at all), especially for merging. Method and author timestamps can be gathered from the commit information, the “version” fill in the monticello.meta directory is not necessary too I think, since we have the commit messages obviously..

I think having a FileTree reader/writer that is not coupled to Monticello would be good, but maybe I’m saying something that is not possible.

> - I get NeoJSON parser errors.

Yes me too now when switching repositories. It appears when you switch from a repository, it uses the same branch name of the old one.

> - I don't get all my repositories (well, in practice allmost none, however I get all the pharo books), so I can't test that metadata-less.

Alright so this is odd.. Martin had the same behavior. Thank you for all your feedback, it’s very helpful.

> Nice to see that nice graph widget put to good use :) But it could have less white space on the left.

True, I didn’t fine tune the small details yet. ;)

> I'm also a taker for a few more comments...

Yes you’re right, some classes lack comments. Gah, so many TODOs.

@ Dale & Thierry:

> Now, how do I contribute if I want to? Where is the repository I should fork on github? I'm also a taker for a few more comments..

> Very interesting ... if you put your project up on github, I would fork it and port it to tODE as this is something that has been on my todo list for awhile:)

I will set up a GitHub repository tomorrow. Right now it’s on SmalltalkHub, but I think it’s a good idea to put it on GitHub (if only for doing versioning with the tool itself). Then you’ll be able to fork etc.

I’ll report back in this thread.

@ Dimitris:

> By the way do you develop this using the latest pharo 5 image and latest pharo vm ?

Yes I do.

The bugs appear differ a lot from user to user.
Some have trouble logging in, some don’t. Some people don’t get all the repositories in the list, some do, etc.

@ Stef:

> - help if you want

Help is always welcome! Great to see that so many people take interest.

So summary TODO list:

- Put the repository on GitHub for everyone to collaborate
- More documentation
- Fix some of those pesky bugs which vary from person to person.

Thierry Goubier

Re: Basic versioning of GitHub repositories from within Pharo

Hi Skip,

2015-11-24 17:30 GMT+01:00 Skip Lentz <[hidden email]>:

Hi again,

I wil try to respond to most of you in this e-mail:

@ Thierry:

> Would you be able to retrieve the same information from github that GitFileTree does when it extract package version history information?

It would be nice to have some less metadata in the repository (or none at all), especially for merging. Method and author timestamps can be gathered from the commit information, the “version” fill in the monticello.meta directory is not necessary too I think, since we have the commit messages obviously..

Well, GitFileTree has all that code:

- Monticello metadata extraction from git logs

- Filetree without Metadata

Some of the information in the meta directory is necessary (pre/post load scripts).

I think having a FileTree reader/writer that is not coupled to Monticello would be good, but maybe I’m saying something that is not possible.

No, it is possible (everything is), but a bit complex for maintenance (how do you track changes and corrections in the FileTree code?).

> - I get NeoJSON parser errors.

Yes me too now when switching repositories. It appears when you switch from a repository, it uses the same branch name of the old one.

Ok.

> - I don't get all my repositories (well, in practice allmost none, however I get all the pharo books), so I can't test that metadata-less.

Alright so this is odd.. Martin had the same behavior. Thank you for all your feedback, it’s very helpful.

Maybe just an effect of the ordering (github gives you a list of repositories, isn't it?)

> Nice to see that nice graph widget put to good use :) But it could have less white space on the left.

True, I didn’t fine tune the small details yet. ;)

> I'm also a taker for a few more comments...

Yes you’re right, some classes lack comments. Gah, so many TODOs.

Welcome to the world of interesting software :)

@ Dale & Thierry:

> Now, how do I contribute if I want to? Where is the repository I should fork on github? I'm also a taker for a few more comments..

> Very interesting ... if you put your project up on github, I would fork it and port it to tODE as this is something that has been on my todo list for awhile:)

I will set up a GitHub repository tomorrow. Right now it’s on SmalltalkHub, but I think it’s a good idea to put it on GitHub (if only for doing versioning with the tool itself). Then you’ll be able to fork etc.

I’ll report back in this thread.

@ Dimitris:

> By the way do you develop this using the latest pharo 5 image and latest pharo vm ?

Yes I do.

The bugs appear differ a lot from user to user.
Some have trouble logging in, some don’t. Some people don’t get all the repositories in the list, some do, etc.

@ Stef:

> - help if you want

Help is always welcome! Great to see that so many people take interest.

So summary TODO list:

- Put the repository on GitHub for everyone to collaborate
- More documentation
- Fix some of those pesky bugs which vary from person to person.

Thierry

Skip Lentz-2

Re: Basic versioning of GitHub repositories from within Pharo

On Nov 24, 2015, at 5:37 PM, Thierry Goubier <[hidden email]> wrote:

Hi Skip,

2015-11-24 17:30 GMT+01:00 Skip Lentz <[hidden email]>:

It would be nice to have some less metadata in the repository (or none at all), especially for merging. Method and author timestamps can be gathered from the commit information, the “version” fill in the monticello.meta directory is not necessary too I think, since we have the commit messages obviously..

Well, GitFileTree has all that code:
- Monticello metadata extraction from git logs
- Filetree without Metadata
Some of the information in the meta directory is necessary (pre/post load scripts).

Where are those pre/post load scripts defined? Does anyone ever use it?

I think having a FileTree reader/writer that is not coupled to Monticello would be good, but maybe I’m saying something that is not possible.

No, it is possible (everything is), but a bit complex for maintenance (how do you track changes and corrections in the FileTree code?).

Of course, the sky is the limit :P. Although I’m not sure I understand what you mean here, could you explain?

Alright so this is odd.. Martin had the same behavior. Thank you for all your feedback, it’s very helpful.

Maybe just an effect of the ordering (github gives you a list of repositories, isn't it?)

Actually now that I think of it, it’s probably due to the fact that when you request a list of something (e.g. list of commits, list of repositories, etc.) the API gives you a paginated result. This means that it will give you the first 25 repositories, and a link to retrieve the next 25, and so on. Pagination still has to be implemented. API documentation about pagination is here: https://developer.github.com/v3/#pagination

Thierry Goubier

Re: Basic versioning of GitHub repositories from within Pharo

2015-11-25 10:39 GMT+01:00 Skip Lentz <[hidden email]>:

On Nov 24, 2015, at 5:37 PM, Thierry Goubier <[hidden email]> wrote:

Hi Skip,

2015-11-24 17:30 GMT+01:00 Skip Lentz <[hidden email]>:

It would be nice to have some less metadata in the repository (or none at all), especially for merging. Method and author timestamps can be gathered from the commit information, the “version” fill in the monticello.meta directory is not necessary too I think, since we have the commit messages obviously..

Well, GitFileTree has all that code:
- Monticello metadata extraction from git logs
- Filetree without Metadata
Some of the information in the meta directory is necessary (pre/post load scripts).

Where are those pre/post load scripts defined? Does anyone ever use it?

You have, in that directory:

- categories (defines the package tags, for example: https://github.com/ThierryGoubier/AltBrowser/blob/pharo5/Alt-Browser.package/monticello.meta/categories.st)

- initializers (but it's usually empty)

- package (the package name: FileTree uses that)

possibly 'preamble.st' 'postscript.st' 'preambleOfRemoval*' 'postscriptOfRemoval*'

Good question, I can't find any of those and I can't even find the code writing that. Dale?

Dependencies (used by SLICEs).

What is conflict-prone with git in that repository:

- version

Now, removing the entire directory for a single file...

You can see the diffs of a metadata-less repository if you look at the recent commits of https://github.com/ThierryGoubier/AltBrowser/commits/pharo5

I think having a FileTree reader/writer that is not coupled to Monticello would be good, but maybe I’m saying something that is not possible.

No, it is possible (everything is), but a bit complex for maintenance (how do you track changes and corrections in the FileTree code?).

Of course, the sky is the limit :P. Although I’m not sure I understand what you mean here, could you explain?

FileTree has a large set of test cases, sample repositories, CI setup, cross-platform knowledge, bug fixes coming from a large group of users (far larger than the Pharo community).

Cutting ourselves from that is counter productive and looks like a case of the NIH syndrome.

Alright so this is odd.. Martin had the same behavior. Thank you for all your feedback, it’s very helpful.

Maybe just an effect of the ordering (github gives you a list of repositories, isn't it?)

Actually now that I think of it, it’s probably due to the fact that when you request a list of something (e.g. list of commits, list of repositories, etc.) the API gives you a paginated result. This means that it will give you the first 25 repositories, and a link to retrieve the next 25, and so on. Pagination still has to be implemented. API documentation about pagination is here: https://developer.github.com/v3/#pagination

Ok, understood.

Ben Coman

Re: Basic versioning of GitHub repositories from within Pharo

On Wed, Nov 25, 2015 at 6:31 PM, Thierry Goubier
<[hidden email]> wrote:

>
>
> 2015-11-25 10:39 GMT+01:00 Skip Lentz <[hidden email]>:
>>
>>
>> On Nov 24, 2015, at 5:37 PM, Thierry Goubier <[hidden email]>
>> wrote:
>>
>> Hi Skip,
>>
>> 2015-11-24 17:30 GMT+01:00 Skip Lentz <[hidden email]>:
>>>
>>>
>>> It would be nice to have some less metadata in the repository (or none at
>>> all), especially for merging. Method and author timestamps can be gathered
>>> from the commit information, the “version” fill in the monticello.meta
>>> directory is not necessary too I think, since we have the commit messages
>>> obviously..
>>
>>
>> Well, GitFileTree has all that code:
>> - Monticello metadata extraction from git logs
>> - Filetree without Metadata
>> Some of the information in the meta directory is necessary (pre/post load
>> scripts).
>>
>>
>> Where are those pre/post load scripts defined? Does anyone ever use it?
>
>
> You have, in that directory:
>
> - categories (defines the package tags, for example:
> https://github.com/ThierryGoubier/AltBrowser/blob/pharo5/Alt-Browser.package/monticello.meta/categories.st)
> - initializers (but it's usually empty)
> - package (the package name: FileTree uses that)
>
> possibly 'preamble.st' 'postscript.st' 'preambleOfRemoval*'
> 'postscriptOfRemoval*'
> Good question, I can't find any of those and I can't even find the code
> writing that. Dale?
>
> Dependencies (used by SLICEs).
>
> What is conflict-prone with git in that repository:
> - version
>
> Now, removing the entire directory for a single file...
>
> You can see the diffs of a metadata-less repository if you look at the
> recent commits of
> https://github.com/ThierryGoubier/AltBrowser/commits/pharo5
>
>>
>>> I think having a FileTree reader/writer that is not coupled to Monticello
>>> would be good, but maybe I’m saying something that is not possible.
>>
>>
>> No, it is possible (everything is), but a bit complex for maintenance (how
>> do you track changes and corrections in the FileTree code?).
>>
>>
>> Of course, the sky is the limit :P. Although I’m not sure I understand
>> what you mean here, could you explain?
>
>
> FileTree has a large set of test cases, sample repositories, CI setup,
> cross-platform knowledge, bug fixes coming from a large group of users (far
> larger than the Pharo community).
>
> Cutting ourselves from that is counter productive and looks like a case of
> the NIH syndrome.

Thats cool. What are the other communities?
cheers -ben