Bitbucket Private Repos Redux
Locked
 
|
I'm trying to set up a private Bitbucket project with SmalltalkCI.
-- The last time I tried to load a private Bitbucket project locally via Metacello, I found out [1] that, because Metacello downloads a zip file manually, SSH credentials are not considered, causing failure to load private projects ([digression] IIRC because even supplying credentials was broken [2] [/digression]. The workaround, which was fine for local development, was to specify a local filetree repo in the baseline. However, in a Bitbucket CI pipeline, the local repo is not there (and the absolute file URL can not be recreated). Has/will anything change to allow remote repos to be loaded via SSH keys? Any other workarounds that I'm not seeing? Thanks! [1] http://forum.world.st/Loading-a-Baseline-of-a-Private-Project-tt4854306.html [2] https://github.com/dalehenrich/metacello-work/pull/368 You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
After a long battle with GitLab, I was able to configure the CI to clone private GL repos into suitable locations for my filetree:// urls. Unfortunately, this method is a drag because I now have to manually clone every dependent project in a CI configuration file. Any ideas are welcome. The only thing I can think of is maybe wrapping the load in a handler which tries to do a `LibC system: 'git clone…'`. Is there anything in the API where I can pass some meta-info into the baseline spec to specify the canonical repo? I'm also not sure this will work because IIUC the CI would have to run Pharo in the same shell as the SSH key setup (e.g. not in a sub-shell)… You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
Bump ;) Can anyone point me to a suitable hook (if one exists) where I can try to insert a fallback where, if a local clone doesn't exist, I can try to clone it and then continue. Less urgently, has there ever been any discussion about inserting arbitrary metadata to the Spec API? For example, in this case, even though we are loading dependencies via FT urls (due to the internal limitation of MetaC that SSH key authentication is not supported*), it would be useful to denote the canonical (remote) repo. * Could this limitation be overcome now that we have (at least in Pharo, not sure about other platforms) libgit support? E.g. Iceberg allows push/pull via ssh-agent keys You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
For SSH I think that platform specific solutions are the right approach and are outside the realm of Metacello itself. The developers are expected to make decisions about repos using platform tools (like iceberg) and then use the lock command to inform Metacello where to find it. For the broader problems I am in favor of Metacello load specs like [1].
You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
The intention is to specify your fallback repo in the baseline and then use a Metacello lock to point to the local clone when it becomes available — a platform specific tool’s responsibility Firstly, are gitlab:// URLs supported? Even if they are, I don't think my user/password authentication was ever integrated, but anyway I'm unwilling to store plain text usernames and passwords in my image. MetaC doesn't yet support ssh-agent keys, right? Although as a workaround I guess I could just override with a lock in every case and at least the canonical remote would be declared in the spec. You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
On 10/18/2017 03:47 PM, Sean DeNigris
wrote:
So why is it not possible for you to simply work with local clones? I don't quite understand why you need to have URLs for installing projects kept in private repositories ... doesn't Iceberg support cloning of repositories from github, bitbucket and gitlab ... if they did, there would be no reason to be asking me for support ... That seems to be the standard case, which makes supporting URLs requiring passwords a bad idea ... I'd really rather not get involved in the vagaries of cross-platform support for SSH operations ... the `github:` URL support was invented to provide a smooth transition for folks to start using GitHub without requiring that all users install git on their local machines ... doesn't iceberg provide a service for creating local clones from URLs? As far as I can tell, it will not be practical for me to invent URLs that don't play well with the local tools and given the rate of change of the local tools I don't want to get into the business of keeping up with the changes made by local tools builders ... It has been my experience that local tool builders are able to adapt their tools to Metacello much easier than it is for me to try to adapt Metacello to each of the local tools ... first it was GitFileTree and Theirry invented a custom set of urls that he supported and now I believe that Iceberg has custom urls as well ... and all of the tools know how to translate the public urls for github and bitbucket into a local clone of some sort ... Gitlab/gittorious was always a problem for adapting to Metacello, because their download fo zip files never have provided the kind of information that I needed to support the same features available from bitbucket and github ... perhaps that has changed ... But now I think that all developers should do work with local clones --- which brings us back to local tool support being the important factor ... Yes .. BTW, I don't consider `Metacello lock` a workaround ... it is as it supposed to be ... the external url is translated (by local tools) into a local clone if one does not already exist (i.e., no lock) and then the lock command makes sure that all other refences refer to the locked local clone ... There is an issue when you have multiple images that want to share the same local clones, but I've been talking about solutions for this for several years[1][2] .. perhaps you only need to wait a couple more years before that capability shows up in your local tools:) Dale [1] https://www.youtube.com/watch?v=QshDlH1ADZQ&list=PLCGAAdUizzH05oEXf-KfKt-F2YqfNHSJq&index=8 [2] https://www.youtube.com/watch?v=Ejmqs0xLvSk&index=25&list=PLCGAAdUizzH06AkHg6_UxZ6QZBgz84yAc -- -- You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
So why is it not possible for you to simply work with local clones? Just for CI on e.g. GitLab. The original reason I wanted to trap the errors was to clone the repo and try again (after standardizing my naming convention).
Agreed. Although for anyone using sqs, ss3, or sthub, aren't all their repo communications handled that way already?! Once I realized this, I ported all my projects to GH, but it breaks down eventually for dependencies of dependencies of...
Hmm, that's an interesting idea. Since I'm not too concerned with backward compatibility for these projects, I might be able to leverage Iceberg, but the question remains: where/how can I hook into MetaC to "clone and retry"? Can/should I just wrap the whole `Metacello new…` with an #on:do: block?
Ah, okay. Good to know. So the only question in this scenario is: if I have all local URLS in the spec, in a CI setting how do I know the remote for dependencies without hand rolling the lock for each dependency for every project? That was the other part of my question about attaching arbitrary metadata to parts of the baseline.
:) You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
On 10/20/2017 07:50 AM, Sean DeNigris
wrote:
okay for CI, local cloning should not be necessary ... and isn't for non-private repos ... travisCI has mechanisms[1] for encrypting secret data in .travis.yml files that can then be used to encode your user/pass combo for use in your application (setting the user/pass in the image, for downloading zip files adding username/pass support to Metacello[2]... [1] https://docs.travis-ci.com/user/encryption-keys/ [2] https://github.com/Metacello/metacello/pull/368 Okay, the preferred hook is to have a platform/tool-specific url that Metacello passes through to the platform for interpretation ... I am definitely willing to add better support for handling custom URLs and perhaps this is the route that we can go ... What I'm imagining is a scheme registration where Metacello allows users to register handlers for a scheme and the contract is that the handler will return a valid filetree url, that Metacello will then use for loading the packages ... This should be relatively straightforward to implement and will make it possible for platform/tools/users to invent their own urls without having to modify Metacello which is the case today ... I think that custom URL handlers would address this as long as we make sure that the project can be locked as part of the handling process ... The secret data could be isolated to the handler logic and use travis-style encryption .. If you think that the URL scheme might make sense, let's get an issue opened for this against Metacello where we can hammer out the details. I've got Metacello work in my near term future, since I am part way through a port of Metacello to the base for GemStone so that commercial customers can use git for their development ... Dale [3] https://github.com/Metacello/metacello -- You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
Re: Bitbucket Private Repos Redux
|
|
Dale Henrichs-3 wrote
-- If you think that the URL scheme might make sense, let's get an issue IIUC you're suggesting that there would be a hook so that I could handle seansprivategitlab:// however I wanted behind the scenes. That sounds intriguing, but how would I ensure that the seansprivategitlab:// support was loaded in the image to be used by MetaC? Also, when you say that cloning isn't necessary, last I checked SSH keys, gitlab, and bitbucket did not seem well-supported by MetaC, so I'm not sure how I would avoid it (or whether I want to; I still don't like the idea of unencrypted credentials in the image, even if they're secure on disk; finally I've been using SmalltalkCI - primarily because I thought Travis was GH-only). Custom URL Support https://github.com/Metacello/metacello/issues/475 You received this message because you are subscribed to the Google Groups "Metacello" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
«
Return to Metacello
|
1 view|%1 views
| Free forum by Nabble | Edit this page |