Bitbucket Private Repos Redux

Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Bitbucket Private Repos Redux

sean.p.denigris
I'm trying to set up a private Bitbucket project with SmalltalkCI. 

The last time I tried to load a private Bitbucket project locally via
Metacello, I found out [1] that, because Metacello downloads a zip file
manually, SSH credentials are not considered, causing failure to load
private projects ([digression] IIRC because even supplying credentials was
broken [2] [/digression]. 

The workaround, which was fine for local development, was to specify a local
filetree repo in the baseline. However, in a Bitbucket CI pipeline, the
local repo is not there (and the absolute file URL can not be recreated).

Has/will anything change to allow remote repos to be loaded via SSH keys?
Any other workarounds that I'm not seeing?

Thanks!

[1] http://forum.world.st/Loading-a-Baseline-of-a-Private-Project-tt4854306.html
[2] https://github.com/dalehenrich/metacello-work/pull/368

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bitbucket Private Repos Redux

sean.p.denigris
However, in a Bitbucket CI pipeline, the
local repo is not there (and the absolute file URL can not be recreated).

After a long battle with GitLab, I was able to configure the CI to clone private GL repos into suitable locations for my filetree:// urls. Unfortunately, this method is a drag because I now have to manually clone every dependent project in a CI configuration file. Any ideas are welcome. The only thing I can think of is maybe wrapping the load in a handler which tries to do a `LibC system: 'git clone…'`. Is there anything in the API where I can pass some meta-info into the baseline spec to specify the canonical repo? I'm also not sure this will work because IIUC the CI would have to run Pharo in the same shell as the SSH key setup (e.g. not in a sub-shell)…

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bitbucket Private Repos Redux

sean.p.denigris
The only thing I can think of is maybe wrapping the load in a handler which tries to do a `LibC system: 'git clone…'`. Is there anything in the API where I can pass some meta-info into the baseline spec to specify the canonical repo?
Bump ;)

Can anyone point me to a suitable hook (if one exists) where I can try to insert a fallback where, if a local clone doesn't exist, I can try to clone it and then continue.

Less urgently, has there ever been any discussion about inserting arbitrary metadata to the Spec API? For example, in this case, even though we are loading dependencies via FT urls (due to the internal limitation of MetaC that SSH key authentication is not supported*), it would be useful to denote the canonical (remote) repo.

* Could this limitation be overcome now that we have (at least in Pharo, not sure about other platforms) libgit support? E.g. Iceberg allows push/pull via ssh-agent keys

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bitbucket Private Repos Redux

Dale Henrichs-3

On Oct 15, 2017, at 6:47 AM, Sean DeNigris <[hidden email]> wrote:

The only thing I can think of is maybe wrapping the load in a handler which tries to do a `LibC system: 'git clone…'`. Is there anything in the API where I can pass some meta-info into the baseline spec to specify the canonical repo?
Bump ;)

Can anyone point me to a suitable hook (if one exists) where I can try to insert a fallback where, if a local clone doesn't exist, I can try to clone it and then continue.

The intention is to specify your fallback repo in the baseline and then use a Metacello lock to point to the local clone when it becomes available — a platform specific tool’s responsibility

Less urgently, has there ever been any discussion about inserting arbitrary metadata to the Spec API? For example, in this case, even though we are loading dependencies via FT urls (due to the internal limitation of MetaC that SSH key authentication is not supported*), it would be useful to denote the canonical (remote) repo.

For SSH I think that platform specific solutions are the right approach and are outside the realm of Metacello itself.  The developers are expected to make decisions about repos using platform tools (like iceberg) and then use the lock command to inform Metacello where to find it.

For the broader problems I am in favor of Metacello load specs like [1].


* Could this limitation be overcome now that we have (at least in Pharo, not sure about other platforms) libgit support? E.g. Iceberg allows push/pull via ssh-agent keys

 

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bitbucket Private Repos Redux

sean.p.denigris
The intention is to specify your fallback repo in the baseline and then use a Metacello lock to point to the local clone when it becomes available — a platform specific tool’s responsibility

IIUC you're saying to specify the remote in the baseline, but AFAICT 
this will not work because it's a private gitlab.com repo.
Firstly, are gitlab:// URLs supported? Even if they are, I don't think my user/password authentication was ever integrated, but anyway I'm unwilling to store plain text usernames and passwords in my image. MetaC doesn't yet support ssh-agent keys, right? Although as a workaround I guess I could jusoverride with a lock in every case and at least the canonical remote woulbe declared in the spec. 

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bitbucket Private Repos Redux

Dale Henrichs-3



On 10/18/2017 03:47 PM, Sean DeNigris wrote:
The intention is to specify your fallback repo in the baseline and then use a Metacello lock to point to the local clone when it becomes available — a platform specific tool’s responsibility

IIUC you're saying to specify the remote in the baseline, but AFAICT 
this will not work because it's a private gitlab.com repo.
So why is it not possible for you to simply work with local clones? I don't quite understand why you need to have URLs for installing projects kept in private repositories ... doesn't Iceberg support cloning of repositories from github, bitbucket and gitlab ... if they did, there would be no reason to be asking me for support ...

Firstly, are gitlab:// URLs supported? Even if they are, I don't think my user/password authentication was ever integrated, but anyway I'm unwilling to store plain text usernames and passwords in my image.
That seems to be the standard case, which makes supporting URLs requiring passwords a bad idea ...
MetaC doesn't yet support ssh-agent keys, right?
I'd really rather not get involved in the vagaries of cross-platform support for SSH operations ... the `github:` URL support was invented to provide a smooth transition for folks to start using GitHub without requiring that all users install git on their local machines ... doesn't iceberg provide a service for creating local clones from URLs?

As far as I can tell, it will not be practical for me to invent URLs that don't play well with the local tools and given the rate of change of the local tools I don't want to get into the business of keeping up with the changes made by local tools builders ... It has been my experience that local tool builders are able to adapt their tools to Metacello much easier than it is for me to try to adapt Metacello to each of the local tools ... first it was GitFileTree and Theirry invented a custom set of urls that he supported and now I believe that Iceberg has custom urls as well ... and all of the tools know how to translate the public urls for github and bitbucket into a local clone of some sort ... Gitlab/gittorious was always a problem for adapting to Metacello, because their download fo zip files never have provided the kind of information that I needed to support the same features available from bitbucket and github ... perhaps that has changed ...

But now I think that all developers should do work with local clones --- which brings us back to local tool support being the important factor ...
Although as a workaround I guess I could jusoverride with a lock in every case and at least the canonical remote woulbe declared in the spec.
Yes .. BTW, I don't consider `Metacello lock` a workaround ... it is as it supposed to be ... the external url is translated (by local tools) into a local clone if one does not already exist (i.e., no lock) and then the lock command makes sure that all other refences refer to the locked local clone ...

There is an issue when you have multiple images that want to share the same local clones, but I've been talking about solutions for this for several years[1][2] .. perhaps you only need to wait a couple more years before that capability shows up in your local tools:)

Dale

[1] https://www.youtube.com/watch?v=QshDlH1ADZQ&list=PLCGAAdUizzH05oEXf-KfKt-F2YqfNHSJq&index=8
[2] https://www.youtube.com/watch?v=Ejmqs0xLvSk&index=25&list=PLCGAAdUizzH06AkHg6_UxZ6QZBgz84yAc
--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Bitbucket Private Repos Redux

sean.p.denigris
So why is it not possible for you to simply work with local clones?
Just for CI on e.g. GitLab. The original reason I wanted to trap the errors was to clone the repo and try again (after standardizing my naming convention).

 
That seems to be the standard case, which makes supporting URLs requiring passwords a bad idea ...
Agreed. Although for anyone using sqs, ss3, or sthub, aren't all their repo communications handled that way already?! Once I realized this, I ported all my projects to GH, but it breaks down eventually for dependencies of dependencies of...

doesn't iceberg provide a service for creating local clones from URLs?
Hmm, that's an interesting idea. Since I'm not too concerned with backward compatibility for these projects, I might be able to leverage Iceberg, but the question remains: where/how can I hook into MetaC to "clone and retry"? Can/should I just wrap the whole `Metacello new…` with an #on:do: block?
 
Yes .. BTW, I don't consider `Metacello lock` a workaround ... it is as it supposed to be ... the external url is translated (by local tools) into a local clone if one does not already exist (i.e., no lock) and then the lock command makes sure that all other refences refer to the locked local clone ...
Ah, okay. Good to know. So the only question in this scenario is: if I have all local URLS in the spec, in a CI setting how do I know the remote for dependencies without hand rolling the lock for each dependency for every project? That was the other part of my question about attaching arbitrary metadata to parts of the baseline.
 
perhaps you only need to wait a couple more years before that capability shows up in your local tools:) 
 :)

--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.