brainstorming about gst-package and fetching remote packages

Hi,

Now that there are some gst projects around (you can check on github),
it may be time to reconsider the monolithic tarball approach.

Paolo wanted to start brainstorming about the idea of fetching remote
packages, so here I go :)

One possibility would be to have a remote repository, and a
gst-something command line tool to fetch and install packages from this
repository.

Maybe a rubygem like tool (simplified?).

Another idea would be to add git/svn capabilities to gst-package.

What do you think?

Cheers!

Nico

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On Thu, 2009-11-05 at 10:24 +0100, Nicolas Petton wrote:
> Hi,

Hi,
>
> Now that there are some gst projects around (you can check on github),
> it may be time to reconsider the monolithic tarball approach.
>
> Paolo wanted to start brainstorming about the idea of fetching remote
> packages, so here I go :)

As I've said to you we need a gst-get tools :

- first we need an easy way to get repositories (i.e. the list of
packages)
- for each repositories do we support multiple protocols : ssh, git,
svn, http(s), ftp, ...

We can begin with a simple tool with a list of git repositories. As
there is the gitorious project we can reuse it.

After it should be pretty easy to extend it with other protocols and
better tools ...
  Cheers,
Gwenael

> Nico
> _______________________________________________
> help-smalltalk mailing list
> [hidden email]
> http://lists.gnu.org/mailman/listinfo/help-smalltalk




_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On 11/05/2009 11:32 AM, Gwenael Casaccio wrote:
> - first we need an easy way to get repositories (i.e. the list of
> packages)

I think there would be two parts to implement:

1) download a package given a URL (could be a http URL to a .star
package, or a git/svn URL to a directory including a package file, or both?)

2) find a package's URL given its name.

For 1, support would be needed in the beginning for svn and git.

For 2, we could support a centralized repository at smalltalk.gnu.org
(like rubygems.org) so the URLs could be standard: gst-package
--download iliad -> gst-package --download
http://smalltalk.gnu.org/project/iliad/repository.  The URLs could
redirect to the "real" URL, e.g. from
http://smalltalk.gnu.org/project/iliad/repository to
git://github.com/NicolasPetton/iliad.git.

Do you think this would be too much of a hack?  Maybe it's better to
return some XML thing from
http://smalltalk.gnu.org/project/iliad/repository and parse it?

A third part would be to update packages on the fly from the remote
repository (especially if it is a git or svn repository).

> - for each repositories do we support multiple protocols : ssh, git,
> svn, http(s), ftp, ...

That would be nice.

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On Thu, 2009-11-05 at 12:11 +0100, Paolo Bonzini wrote:
So we'll have a central repository at: http://smalltalk.gnu.org/

For each URLs inside http://smalltalk.gnu.org/packages/ we will have a
package.(st|xml) file that stores the information about the related
project (website, version, ...)

that seems fine for me.

> A third part would be to update packages on the fly from the remote
> repository (especially if it is a git or svn repository).

Yes even if it is not a git or svn repository we can download new
version.




_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On Thu, 2009-11-05 at 14:20 +0100, Gwenael Casaccio wrote:
Maybe with :

libcurl is a free and easy-to-use client-side URL transfer library,
supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP,
LDAPS and FILE. libcurl supports SSL certificates, HTTP POST, HTTP PUT,
FTP uploading, HTTP form based upload, proxies, cookies, user+password
authentication (Basic, Digest, NTLM, Negotiate, Kerberos4), file
transfer resume, http proxy tunneling and more!

libcurl is highly portable, it builds and works identically on numerous
platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HPUX,
IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOs, Mac
OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS and more...

libcurl is free, thread-safe, IPv6 compatible, feature rich, well
supported, fast, thoroughly documented and is already used by many
known, big and successful companies and numerous applications.

;)




_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On 05.11.2009, at 14:11, Paolo Bonzini wrote:

> Do you think this would be too much of a hack?  Maybe it's better to  
> return some XML thing from http://smalltalk.gnu.org/project/iliad/repository 
>  and parse it?

Maybe JSON instead of XML?
It's more readable and more compact.


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

> Maybe with libcurl
>
> ;)

Having bindings to libcurl is surely a good thing for the future, but
there is already an HTTP client in package NetClients.

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On 11/05/2009 02:20 PM, Gwenael Casaccio wrote:
> So we'll have a central repository at:http://smalltalk.gnu.org/
>
> For each URLs insidehttp://smalltalk.gnu.org/packages/  we will have a
> package.(st|xml) file that stores the information about the related
> project (website, version, ...)
>
> that seems fine for me.

Right, having a package.xml file directly on smalltalk.gnu.org seems
better than having yet another XML format.  On the other hand we need to
have smalltalk.gnu.org as the database only---the files need to be
stored elsewhere.  So I pushed two changes:

1) to gst, adding an <url> element to package.xml that will be used to
fetch the actual contents from a git or svn repository (or similar, you
got the idea).

2) to the smalltalk.gnu.org site, adding a "package.xml" field to
projects that can be used to give the address of a package.xml file.
See http://smalltalk.gnu.org/project/sandstonedb/package.xml for example
(sorry Nico for abusing my admin privileges :-) and modifying your package).


For example, to "register" Iliad the procedure is as follows:

1) add a <url>git://github.com/NicolasPetton/iliad.git</git> element to
his package.xml; the latest PackageBuilder.st in the gst repo does
include the possibility to create an url tag.  In fact, you are all
welcome to add <url> tags to your package.xml files even now (except for
VisualGST).

2) publish the package.xml file in his git repo (it doesn't matter if it
is automatically generated, it is for greater good!...);

3) go to smalltalk.gnu.org and save
http://github.com/NicolasPetton/iliad/raw/master/package.xml as the
package.xml file for Iliad.

Now, http://smalltalk.gnu.org/project/Iliad/package.xml will magically
include a git:// URL that gst-package could use to clone the repository,
make an Iliad.star file and install it.

All that's left is to remove the last "could"! :-)

Paolo



_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

> sorry Nico for abusing my admin privileges :-) and modifying your package).

Hehe, no problem :)
That's just great! Thanks Paolo, you came up with a very solution :)

Cheers!

Nico

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

> That's just great! Thanks Paolo, you came up with a very solution :)
I meant very *nice* solution

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

On 11/06/2009 12:02 AM, Nicolas Petton wrote:
>
>> That's just great! Thanks Paolo, you came up with a very solution :)
> I meant very *nice* solution

Well, I would have come up with a hack if it wasn't for Gwen's reply...
  I was almost going to implement a redirect from http://blah to
git://something -- this one is much nicer indeed.

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk