How do I convert a FileTree project to Metadataless GitFileTree project?
Locked
12
12
How do I convert a FileTree project to Metadataless GitFileTree project?
 
|
Thierry, I think I tried the obvious:
I even tried removing and adding the repo after editting the .filetree file .... I have a handful of additional random things that I will try, but I think it is also time to find out the proper technique:) I want to gain experience with GitFileTree; test out the Metacello Cypress mode; and perhaps honor the `"Metadata" : "false"` in the GemStone version of FileTree ... With a short time of usage, I would really like to be able to save all of the packages in a project (including the BaselineOf) in one commit ... I often have multi-package units of work that really should be committed together and of course if a method is moved across a package boundary, the independent commits don't allow git to recognize the move as a "rename" ... Also I'm hoping that in metadataless mode I stop scanning the entire known repository universe for package versions that will not be found anywhere but in the repositories that I've explicitly associated with the the package itself:) Thanks, Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
Hi Dale,
Le 28/06/2016 20:49, Dale Henrichs a écrit : > Thierry, > > I think I tried the obvious: > > 1. open existing repository with gitfiletree:// > 2. reload all packages from gitfiletree repo and remove old filetree:// > repo from each package > 3. edit .filetree and add `"Metadata" : "false"` > > but when I save a package I continue to get metadata created ... With that sequence, I'm sure the gitfiletree repository in memory has still kept the Metadata : true property, and so hasn't taken in account the on-disk change. Like FileTree, it only loads the repository properties on the very first access to that repository. > I even tried removing and adding the repo after editting the .filetree > file .... And that didn't work ? I suspected it would... unless the gitfiletree repository of step 1 above was not garbage collected. > I have a handful of additional random things that I will try, but I > think it is also time to find out the proper technique:) I haven't provided an API for the switch, really. And I don't know where to add that option inside the Monticello gui. What I did, in the few cases I did it, was to inspect the in-image MCFileTreeGitRepository and change the property in the inspector ;) > I want to gain experience with GitFileTree; test out the Metacello > Cypress mode; and perhaps honor the `"Metadata" : "false"` in the > GemStone version of FileTree ... > > With a short time of usage, I would really like to be able to save all > of the packages in a project (including the BaselineOf) in one commit > ... I often have multi-package units of work that really should be > committed together and of course if a method is moved across a package > boundary, the independent commits don't allow git to recognize the move > as a "rename" ... We would need a specific API for that. At the moment what would work is to save the baselineOf with all the baseline packages as dependencies. Now, I'd like to be able to query the baseline to know which packages it describes, and save all of them in one go. Another possibility would be to add an option with a no-commit flag (write but do not commit, do not try to update the package version) and add a button to the window appearing after each written package version to do the commit, store temporarily all such "staged" packages info in the repository object, and provide a warning if one tries to resave a package before a commit... > Also I'm hoping that in metadataless mode I stop scanning the entire > known repository universe for package versions that will not be found > anywhere but in the repositories that I've explicitly associated with > the the package itself:) I don't think it happens there. MCGitFileTreeVersionInfo already tracks the repository it comes from, but I'm pretty sure that MC doesn't ask the version info for the new data. Maybe add a new type of WorkingCopy object? Thierry > Thanks, > > Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
Thierry,
I don't think I can do it at all... I've deleted all of the packages, I've removed all traces of the repository and when I add the repository and inspect I get hasMetadata false... I then deleted the metadata directory from every package on disk and committed ... I then loaded the packages again from disk and made a change and when I commit the change version number increments and voila, I have a new monticello meta data directoy on disk free of charge ... and yes hasMetadata is still false ... I looked at the code and it seems that you are extracting the meta data from the git repo (once you have meta data you must have meta data forever?) It looks like it is not possible to switch to metadataless which is a shame ... all of the repos where I'd use this have monticello meta data in the history so you'll find it if you go looking for it:) additional comments in-lined below ... On 6/28/16 12:19 PM, Thierry Goubier wrote: > Hi Dale, > > Le 28/06/2016 20:49, Dale Henrichs a écrit : >> Thierry, >> >> I think I tried the obvious: >> >> 1. open existing repository with gitfiletree:// >> 2. reload all packages from gitfiletree repo and remove old filetree:// >> repo from each package >> 3. edit .filetree and add `"Metadata" : "false"` >> >> but when I save a package I continue to get metadata created ... > > With that sequence, I'm sure the gitfiletree repository in memory has > still kept the Metadata : true property, and so hasn't taken in > account the on-disk change. Like FileTree, it only loads the > repository properties on the very first access to that repository. > >> I even tried removing and adding the repo after editting the .filetree >> file .... > > And that didn't work ? I suspected it would... unless the gitfiletree > repository of step 1 above was not garbage collected. > >> I have a handful of additional random things that I will try, but I >> think it is also time to find out the proper technique:) > > I haven't provided an API for the switch, really. And I don't know > where to add that option inside the Monticello gui. although it looks like the issue is deeper than simply changing the metadata state:( > > What I did, in the few cases I did it, was to inspect the in-image > MCFileTreeGitRepository and change the property in the inspector ;) > >> I want to gain experience with GitFileTree; test out the Metacello >> Cypress mode; and perhaps honor the `"Metadata" : "false"` in the >> GemStone version of FileTree ... >> >> With a short time of usage, I would really like to be able to save all >> of the packages in a project (including the BaselineOf) in one commit >> ... I often have multi-package units of work that really should be >> committed together and of course if a method is moved across a package >> boundary, the independent commits don't allow git to recognize the move >> as a "rename" ... > > We would need a specific API for that. > > At the moment what would work is to save the baselineOf with all the > baseline packages as dependencies. unless there have been radical changes to the way dependencies work, this would defeat the purpose of having a baseline, i.e., all packages get loaded when the baseline itself is loaded ... > > Now, I'd like to be able to query the baseline to know which packages > it describes, and save all of them in one go. I have this feature implemented in tODE ... of course in tODE I have created a Metacello Project Browser that is used instead of a Monticello Package Browser ... tODE still have the ability to manipulate packages independent of the project, but the vast majority of the work is done by loading and saving projects instead of packages ... The upshot is that I have code that supports all of the necessary functionality, but someone woule need to build a Metacello Project Browser in Pharo to make this funcitonality available to users --- and I don't have the bandwidth for that ... > > Another possibility would be to add an option with a no-commit flag > (write but do not commit, do not try to update the package version) > and add a button to the window appearing after each written package > version to do the commit, store temporarily all such "staged" packages > info in the repository object, and provide a warning if one tries to > resave a package before a commit... A poor man's project browser would have some additional menu items for the BaselineOf package: save project and load project would probably be sufficient. The you would go around a write all of the dirty packages associated with the baseline ... in addition to the baseline itself ... and then do your git commit when completed ... > >> Also I'm hoping that in metadataless mode I stop scanning the entire >> known repository universe for package versions that will not be found >> anywhere but in the repositories that I've explicitly associated with >> the the package itself:) > > I don't think it happens there. MCGitFileTreeVersionInfo already > tracks the repository it comes from, but I'm pretty sure that MC > doesn't ask the version info for the new data. Maybe add a new type of > WorkingCopy object? When working with a metadataless repository why would need to look up version info for the package in other repository? In the past, the Monticello Browser would only look in the repositories of the repository group for a package to look for package versions ... now it seems that all of the repositoris in the default repository group are scanned for versions of the package being committed. Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
BTW, I loaded git file tree with the Catalog Browser, so I'm not
necessarily using the latest code from Github: BaselineOfGitFileTree-ThierryGoubier.16 Dale On 6/28/16 12:55 PM, Dale Henrichs wrote: > Thierry, > > I don't think I can do it at all... I've deleted all of the packages, > I've removed all traces of the repository and when I add the > repository and inspect I get hasMetadata false... > > I then deleted the metadata directory from every package on disk and > committed ... I then loaded the packages again from disk and made a > change and when I commit the change version number increments and > voila, I have a new monticello meta data directoy on disk free of > charge ... and yes hasMetadata is still false ... > > I looked at the code and it seems that you are extracting the meta > data from the git repo (once you have meta data you must have meta > data forever?) > > It looks like it is not possible to switch to metadataless which is a > shame ... all of the repos where I'd use this have monticello meta > data in the history so you'll find it if you go looking for it:) > > > additional comments in-lined below ... > > On 6/28/16 12:19 PM, Thierry Goubier wrote: >> Hi Dale, >> >> Le 28/06/2016 20:49, Dale Henrichs a écrit : >>> Thierry, >>> >>> I think I tried the obvious: >>> >>> 1. open existing repository with gitfiletree:// >>> 2. reload all packages from gitfiletree repo and remove old >>> filetree:// >>> repo from each package >>> 3. edit .filetree and add `"Metadata" : "false"` >>> >>> but when I save a package I continue to get metadata created ... >> >> With that sequence, I'm sure the gitfiletree repository in memory has >> still kept the Metadata : true property, and so hasn't taken in >> account the on-disk change. Like FileTree, it only loads the >> repository properties on the very first access to that repository. >> >>> I even tried removing and adding the repo after editting the .filetree >>> file .... >> >> And that didn't work ? I suspected it would... unless the gitfiletree >> repository of step 1 above was not garbage collected. >> >>> I have a handful of additional random things that I will try, but I >>> think it is also time to find out the proper technique:) >> >> I haven't provided an API for the switch, really. And I don't know >> where to add that option inside the Monticello gui. > a Smalltalk expression or setting woudl be better than nothing ... > although it looks like the issue is deeper than simply changing the > metadata state:( >> >> What I did, in the few cases I did it, was to inspect the in-image >> MCFileTreeGitRepository and change the property in the inspector ;) >> >>> I want to gain experience with GitFileTree; test out the Metacello >>> Cypress mode; and perhaps honor the `"Metadata" : "false"` in the >>> GemStone version of FileTree ... >>> >>> With a short time of usage, I would really like to be able to save all >>> of the packages in a project (including the BaselineOf) in one commit >>> ... I often have multi-package units of work that really should be >>> committed together and of course if a method is moved across a package >>> boundary, the independent commits don't allow git to recognize the move >>> as a "rename" ... >> >> We would need a specific API for that. > yep.. >> >> At the moment what would work is to save the baselineOf with all the >> baseline packages as dependencies. > unless there have been radical changes to the way dependencies work, > this would defeat the purpose of having a baseline, i.e., all packages > get loaded when the baseline itself is loaded ... >> >> Now, I'd like to be able to query the baseline to know which packages >> it describes, and save all of them in one go. > I have this feature implemented in tODE ... of course in tODE I have > created a Metacello Project Browser that is used instead of a > Monticello Package Browser ... tODE still have the ability to > manipulate packages independent of the project, but the vast majority > of the work is done by loading and saving projects instead of packages > ... > > The upshot is that I have code that supports all of the necessary > functionality, but someone woule need to build a Metacello Project > Browser in Pharo to make this funcitonality available to users --- and > I don't have the bandwidth for that ... >> >> Another possibility would be to add an option with a no-commit flag >> (write but do not commit, do not try to update the package version) >> and add a button to the window appearing after each written package >> version to do the commit, store temporarily all such "staged" >> packages info in the repository object, and provide a warning if one >> tries to resave a package before a commit... > A poor man's project browser would have some additional menu items for > the BaselineOf package: save project and load project would probably > be sufficient. The you would go around a write all of the dirty > packages associated with the baseline ... in addition to the baseline > itself ... and then do your git commit when completed ... >> >>> Also I'm hoping that in metadataless mode I stop scanning the entire >>> known repository universe for package versions that will not be found >>> anywhere but in the repositories that I've explicitly associated with >>> the the package itself:) >> >> I don't think it happens there. MCGitFileTreeVersionInfo already >> tracks the repository it comes from, but I'm pretty sure that MC >> doesn't ask the version info for the new data. Maybe add a new type >> of WorkingCopy object? > > When working with a metadataless repository why would need to look up > version info for the package in other repository? > > In the past, the Monticello Browser would only look in the > repositories of the repository group for a package to look for package > versions ... now it seems that all of the repositoris in the default > repository group are scanned for versions of the package being committed. > > Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Thierry Goubier
On 6/28/16 12:19 PM, Thierry Goubier wrote: > Hi Dale, > > Le 28/06/2016 20:49, Dale Henrichs a écrit : > >> Also I'm hoping that in metadataless mode I stop scanning the entire >> known repository universe for package versions that will not be found >> anywhere but in the repositories that I've explicitly associated with >> the the package itself:) > > I don't think it happens there. MCGitFileTreeVersionInfo already > tracks the repository it comes from, but I'm pretty sure that MC > doesn't ask the version info for the new data. Maybe add a new type of > WorkingCopy object? Thierry, In Pharo5.0 it seems that the Pharo inbox and several other repositories get scanned whenever I do a package diff or do a package save from the Monticello Browser ... a few weeks ago, you responded to someone elses complaint about this annoying behavior and at this point (after reading through a bunch of code trying to get some understanding about why metadataless Git Filetree is not working for me) I believe that you were claiming responsibility for a problem that is not yours ... It is true that on every save, you do a refresh for the GitFileTree and it is true that this is slightly annoying (but understandable), but the truth is that I am seeing this annoying inbox and friends scanning when I don't have git file tree loaded in the image ... Soooo, my crack about "scanning the entire known universe" was aimed at the behavior I see without GitFileTree loaded in the image .... Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Dale Henrichs-3
Dale,
I'm sure it is possible. Wait, wait! If you have in your .filetree "Metadata" : "false" then this is fine and it has switched to the metadata-less mode. To see the changes on disk, you need to save a new version of your packages, that should be all. I just tried and that works. 1- remove the GitFileTree repository from your image 2- write the property "Metadata" : "false" in the .filetree on disk 3- re-add the GitFileTree repository (local) 4- modify then save one of the repository packages 5- look on disk: no more monticello.meta/version! Note that I had no packages in the image linked to that repository at 1-, because I'm not sure the simple removal would have really removed the repository singleton object. Le 28/06/2016 21:55, Dale Henrichs a écrit : > Thierry, > > I don't think I can do it at all... I've deleted all of the packages, > I've removed all traces of the repository and when I add the repository > and inspect I get hasMetadata false... > > I then deleted the metadata directory from every package on disk and > committed ... I then loaded the packages again from disk and made a > change and when I commit the change version number increments and voila, > I have a new monticello meta data directoy on disk free of charge ... > and yes hasMetadata is still false ... > > I looked at the code and it seems that you are extracting the meta data > from the git repo (once you have meta data you must have meta data > forever?) > > It looks like it is not possible to switch to metadataless which is a > shame ... all of the repos where I'd use this have monticello meta data > in the history so you'll find it if you go looking for it:) This is a vcs, so, anyway, the monticello meta data will stay in the history; you just have to checkout an old enough commit to get it :) > additional comments in-lined below ... More comments below as well. > On 6/28/16 12:19 PM, Thierry Goubier wrote: >> Hi Dale, >> >> Le 28/06/2016 20:49, Dale Henrichs a écrit : >>> Thierry, >>> >>> I think I tried the obvious: >>> >>> 1. open existing repository with gitfiletree:// >>> 2. reload all packages from gitfiletree repo and remove old filetree:// >>> repo from each package >>> 3. edit .filetree and add `"Metadata" : "false"` >>> >>> but when I save a package I continue to get metadata created ... >> >> With that sequence, I'm sure the gitfiletree repository in memory has >> still kept the Metadata : true property, and so hasn't taken in >> account the on-disk change. Like FileTree, it only loads the >> repository properties on the very first access to that repository. >> >>> I even tried removing and adding the repo after editting the .filetree >>> file .... >> >> And that didn't work ? I suspected it would... unless the gitfiletree >> repository of step 1 above was not garbage collected. >> >>> I have a handful of additional random things that I will try, but I >>> think it is also time to find out the proper technique:) >> >> I haven't provided an API for the switch, really. And I don't know >> where to add that option inside the Monticello gui. > a Smalltalk expression or setting woudl be better than nothing ... > although it looks like the issue is deeper than simply changing the > metadata state:( No, this was just that. >> What I did, in the few cases I did it, was to inspect the in-image >> MCFileTreeGitRepository and change the property in the inspector ;) >> >>> I want to gain experience with GitFileTree; test out the Metacello >>> Cypress mode; and perhaps honor the `"Metadata" : "false"` in the >>> GemStone version of FileTree ... >>> >>> With a short time of usage, I would really like to be able to save all >>> of the packages in a project (including the BaselineOf) in one commit >>> ... I often have multi-package units of work that really should be >>> committed together and of course if a method is moved across a package >>> boundary, the independent commits don't allow git to recognize the move >>> as a "rename" ... >> >> We would need a specific API for that. > yep.. >> >> At the moment what would work is to save the baselineOf with all the >> baseline packages as dependencies. > unless there have been radical changes to the way dependencies work, > this would defeat the purpose of having a baseline, i.e., all packages > get loaded when the baseline itself is loaded ... Yes, but this is the only point where a MC repository sees a bunch of packages to write to disk and not a single package. >> Now, I'd like to be able to query the baseline to know which packages >> it describes, and save all of them in one go. > I have this feature implemented in tODE ... of course in tODE I have > created a Metacello Project Browser that is used instead of a Monticello > Package Browser ... tODE still have the ability to manipulate packages > independent of the project, but the vast majority of the work is done by > loading and saving projects instead of packages ... > > The upshot is that I have code that supports all of the necessary > functionality, but someone woule need to build a Metacello Project > Browser in Pharo to make this funcitonality available to users --- and I > don't have the bandwidth for that ... So, what we would need is a project concept, and packages being loaded via a baseline are added to that project, and, when saving, you manipulate that project ? Stef, I seem to remember you have planned for someone to work on Git integration. I think this would be about the right feature to add, combined with versionner... >> >> Another possibility would be to add an option with a no-commit flag >> (write but do not commit, do not try to update the package version) >> and add a button to the window appearing after each written package >> version to do the commit, store temporarily all such "staged" packages >> info in the repository object, and provide a warning if one tries to >> resave a package before a commit... > A poor man's project browser would have some additional menu items for > the BaselineOf package: save project and load project would probably be > sufficient. The you would go around a write all of the dirty packages > associated with the baseline ... in addition to the baseline itself ... > and then do your git commit when completed ... >> >>> Also I'm hoping that in metadataless mode I stop scanning the entire >>> known repository universe for package versions that will not be found >>> anywhere but in the repositories that I've explicitly associated with >>> the the package itself:) >> >> I don't think it happens there. MCGitFileTreeVersionInfo already >> tracks the repository it comes from, but I'm pretty sure that MC >> doesn't ask the version info for the new data. Maybe add a new type of >> WorkingCopy object? > > When working with a metadataless repository why would need to look up > version info for the package in other repository? > > In the past, the Monticello Browser would only look in the repositories > of the repository group for a package to look for package versions ... > now it seems that all of the repositoris in the default repository group > are scanned for versions of the package being committed. I'll have a look and see if I can cut that in the case of GitFileTree packages. Thierry > Dale > > |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
On 6/28/16 2:16 PM, Thierry Goubier wrote: > Dale, > > I'm sure it is possible. Wait, wait! If you have in your .filetree > "Metadata" : "false" then this is fine and it has switched to the > metadata-less mode. > > To see the changes on disk, you need to save a new version of your > packages, that should be all. > > I just tried and that works. > > 1- remove the GitFileTree repository from your image > 2- write the property "Metadata" : "false" in the .filetree on disk > 3- re-add the GitFileTree repository (local) > 4- modify then save one of the repository packages > 5- look on disk: no more monticello.meta/version! > > Note that I had no packages in the image linked to that repository at > 1-, because I'm not sure the simple removal would have really removed > the repository singleton object. and of course I looked very carefully to ensure that the packages were gone and the repository was not referenced anywhere... I spent a fair amount of time reading the git filetree code to try to understand who is creating the monticello.meta directory and reading all of your code, I could not find any obvious culprits.... so I don't think that your code is directly responsible for the creation of the metadata dir... However, I just sprayed a bunch of self halts into the code in an attempt to find out who might be causing the directory to be created and I tripped across MCLazyVersionInfo who I think is the culprit for the "scanning the entire known universe" and _might_ be the culprit for "lazily" creating the "missing" monticello.meta directory - I haven't gotten that far yet. With regards to "scanning the entire known universe". It looks like any sender of #ancestors (which is not an unusual thing during save --- my current case or diff another case where I've seen the full scan) to an instance MCLazyVersionInfo can end up doing following: MCRepositoryGroup default in: [ :r | r versionWithInfo: self ifNone: [ ] ] which literally scans the entire repository universe ... This is part of the method #loadAncestorsAndStepChildren which is documented to: "This code scans all the repositories to find the exact package version and reload the relevant info." In my image I have 475 instances of MCLazyVersionInfo and 481 instances of MCWorkingCopy (without knowing for certain, I'll speculate that the 6 working copies without MCLazyVersionInfo are the 6 packages that are instances of MCGitFileTreeVersionInfo a subclass of MCLazyVersionInfo) ... The MCLazyVersionInfo caches the result of the full universe scan in a WeakArray ... so if there's an intervening gc (or two) between the time that one saves a package, the full repository scan will be re-run again ... and again ... and again .. So I think I now understand why I see so many scans of the pharo inbox and presumably all of the repos almost every time I save a package ... not very friendly... This scan was triggered when I tried to save one of the packages that I had just loaded and in this case the MCGitFileTreeVersionInfo triggered a scan because the repo instance variable was nil ... perhaps this is the difference between your run and mine ... in my case the repo was not set correctly and in your case it was? Hopefully someone will do something about this "lazy version info problem" personally I would gladly trade off some memory and disk consumption to avoid having to wait for the entire repository universe to be scanned every time I save a package ...perhaps there is a setting somewhere? I will continue trying to work through meta data directory creation problem and see if I can pinpoint the metadata dir creation culprit ... right now I know that the packages are loaded without creating the directory ... Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Thierry Goubier
On 6/28/16 2:16 PM, Thierry Goubier wrote: > Dale, > > I'm sure it is possible. Wait, wait! If you have in your .filetree > "Metadata" : "false" then this is fine and it has switched to the > metadata-less mode. > > To see the changes on disk, you need to save a new version of your > packages, that should be all. > > I just tried and that works. > > 1- remove the GitFileTree repository from your image > 2- write the property "Metadata" : "false" in the .filetree on disk > 3- re-add the GitFileTree repository (local) > 4- modify then save one of the repository packages > 5- look on disk: no more monticello.meta/version! > > Note that I had no packages in the image linked to that repository at > 1-, because I'm not sure the simple removal would have really removed > the repository singleton object. > > Le 28/06/2016 21:55, Dale Henrichs a écrit : >> Thierry, >> >> I don't think I can do it at all... I've deleted all of the packages, >> I've removed all traces of the repository and when I add the repository >> and inspect I get hasMetadata false... >> >> I then deleted the metadata directory from every package on disk and >> committed ... I then loaded the packages again from disk and made a >> change and when I commit the change version number increments and voila, >> I have a new monticello meta data directoy on disk free of charge ... >> and yes hasMetadata is still false ... >> >> I looked at the code and it seems that you are extracting the meta data >> from the git repo (once you have meta data you must have meta data >> forever?) >> >> It looks like it is not possible to switch to metadataless which is a >> shame ... all of the repos where I'd use this have monticello meta data >> in the history so you'll find it if you go looking for it:) > > This is a vcs, so, anyway, the monticello meta data will stay in the > history; you just have to checkout an old enough commit to get it :) > >> additional comments in-lined below ... > > More comments below as well. > >> On 6/28/16 12:19 PM, Thierry Goubier wrote: >>> Hi Dale, >>>> >>>> At the moment what would work is to save the baselineOf with all the >>>> baseline packages as dependencies. >>> unless there have been radical changes to the way dependencies work, >>> this would defeat the purpose of having a baseline, i.e., all packages >>> get loaded when the baseline itself is loaded ... > > Yes, but this is the only point where a MC repository sees a bunch of > packages to write to disk and not a single package. project this is not a viable alternative... > >>> Now, I'd like to be able to query the baseline to know which packages >>> it describes, and save all of them in one go. >> I have this feature implemented in tODE ... of course in tODE I have >> created a Metacello Project Browser that is used instead of a Monticello >> Package Browser ... tODE still have the ability to manipulate packages >> independent of the project, but the vast majority of the work is done by >> loading and saving projects instead of packages ... >> >> The upshot is that I have code that supports all of the necessary >> functionality, but someone woule need to build a Metacello Project >> Browser in Pharo to make this funcitonality available to users --- and I >> don't have the bandwidth for that ... > > So, what we would need is a project concept, and packages being loaded > via a baseline are added to that project, and, when saving, you > manipulate that project ? configurations) and using Metacello it is possible to determine which pacakges belong to which projects etc. so it is not necessary to "register packages with a project" ... the project registry in Metacello is enough ... > > Stef, I seem to remember you have planned for someone to work on Git > integration. I think this would be about the right feature to add, > combined with versionner... Yep I agree ... and as I have often said, I've written code in tODE that uses Metacello to manage projects with a host of git support features as well ... and I would gladly offer advice and assistance to help someone adapt the code/lessons that I've larned with tODE to Pharo ... I have built a definition model upon which a number of the tools are based, but it is probably time for a next generation implementation ... > >>> >>> Another possibility would be to add an option with a no-commit flag >>> (write but do not commit, do not try to update the package version) >>> and add a button to the window appearing after each written package >>> version to do the commit, store temporarily all such "staged" packages >>> info in the repository object, and provide a warning if one tries to >>> resave a package before a commit... >> A poor man's project browser would have some additional menu items for >> the BaselineOf package: save project and load project would probably be >> sufficient. The you would go around a write all of the dirty packages >> associated with the baseline ... in addition to the baseline itself ... >> and then do your git commit when completed ... >>> >>>> Also I'm hoping that in metadataless mode I stop scanning the entire >>>> known repository universe for package versions that will not be found >>>> anywhere but in the repositories that I've explicitly associated with >>>> the the package itself:) >>> >>> I don't think it happens there. MCGitFileTreeVersionInfo already >>> tracks the repository it comes from, but I'm pretty sure that MC >>> doesn't ask the version info for the new data. Maybe add a new type of >>> WorkingCopy object? >> >> When working with a metadataless repository why would need to look up >> version info for the package in other repository? >> >> In the past, the Monticello Browser would only look in the repositories >> of the repository group for a package to look for package versions ... >> now it seems that all of the repositoris in the default repository group >> are scanned for versions of the package being committed. > > I'll have a look and see if I can cut that in the case of GitFileTree > packages. > while reading the code I recognized that many of the basic patterns were exactly as I had remembered them from years ago ... however ... MCLazyVersionInfo this puppy with its "default behavior" to scan the universe is the real culprit ... I would think that at a minimum the repository or repository group would/could be know at the time that the MCLazyVersionInfo was created and a scan of just those repositories --- already associated with the project --- would not be nearly as bad as when we have now ... Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Thierry Goubier
On 6/28/16 2:16 PM, Thierry Goubier wrote: > Dale, > > I'm sure it is possible. Wait, wait! If you have in your .filetree > "Metadata" : "false" then this is fine and it has switched to the > metadata-less mode. > > To see the changes on disk, you need to save a new version of your > packages, that should be all. > > I just tried and that works. > > 1- remove the GitFileTree repository from your image > 2- write the property "Metadata" : "false" in the .filetree on disk > 3- re-add the GitFileTree repository (local) > 4- modify then save one of the repository packages > 5- look on disk: no more monticello.meta/version! > > Note that I had no packages in the image linked to that repository at > 1-, because I'm not sure the simple removal would have really removed > the repository singleton object. ensures that monticello.meta directory exists... Of course as I reread your comment "no more monticello.meta/version" I will have to say that I've only been looking at whether or not the monticello.meta directory existed or not ... I equate "no monticello meta data" as no monticello.meta directory and you interpret it as "no monticallo version file" ... well I can live with that ... I swore that I saw version files being updated in some of my experiments ... but now that I know that I should only look at the monticello.meta/version file, I will try yet again ... Thanks for you patience and help ... Dale |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
Thierry,
Okay ... it is "working" now ... I was also misled by the fact that you are continuing to fabricate Monticello version numbers which presumably cannot be relied upon in any way. Tugrik-Help-DaleHenrichs.11 will show up in each branch that includes the commit for "Tugrik-Help-DaleHenrichs.10", but the SHA and contents would be different for each one of the "Tugrik-Help-DaleHenrichs.11" , is that right? Perhaps using the short SHA in place of the "version number" would be safer and provide useful information in the version number slot? If I support "Metadata" : "false" in GemStone, I do not intend to fabricate a "realistic looking Monticello version number" ... but I will look into using the short SHA (when in a git repo) and perhaps fall back to cypress.1 for non-git repos... Anyway, I will now be able to move forward with my Metacello Cypress experiments and also try to understand how Metacello loads are affected by metadtaless, since you _are_ fabricating Monticello version numbers, my previous assumptions are not correct ... Thanks again! Dale On 6/28/16 4:11 PM, Dale Henrichs wrote: > > > On 6/28/16 2:16 PM, Thierry Goubier wrote: >> Dale, >> >> I'm sure it is possible. Wait, wait! If you have in your .filetree >> "Metadata" : "false" then this is fine and it has switched to the >> metadata-less mode. >> >> To see the changes on disk, you need to save a new version of your >> packages, that should be all. >> >> I just tried and that works. >> >> 1- remove the GitFileTree repository from your image >> 2- write the property "Metadata" : "false" in the .filetree on disk >> 3- re-add the GitFileTree repository (local) >> 4- modify then save one of the repository packages >> 5- look on disk: no more monticello.meta/version! >> >> Note that I had no packages in the image linked to that repository at >> 1-, because I'm not sure the simple removal would have really removed >> the repository singleton object. > Well I've found the culprit: MCFileTreeWriter>>addString:at:encodedTo: > ensures that monticello.meta directory exists... > > Of course as I reread your comment "no more monticello.meta/version" I > will have to say that I've only been looking at whether or not the > monticello.meta directory existed or not ... I equate "no monticello > meta data" as no monticello.meta directory and you interpret it as "no > monticallo version file" ... well I can live with that ... I swore > that I saw version files being updated in some of my experiments ... > but now that I know that I should only look at the > monticello.meta/version file, I will try yet again ... > > Thanks for you patience and help ... > > Dale > |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
Hi Dale,
Le 29/06/2016 01:50, Dale Henrichs a écrit : > Thierry, > > Okay ... it is "working" now ... I was also misled by the fact that you > are continuing to fabricate Monticello version numbers which presumably > cannot be relied upon in any way. > > Tugrik-Help-DaleHenrichs.11 will show up in each branch that includes > the commit for "Tugrik-Help-DaleHenrichs.10", but the SHA and contents > would be different for each one of the "Tugrik-Help-DaleHenrichs.11" , > is that right? Yes. > Perhaps using the short SHA in place of the "version number" would be > safer and provide useful information in the version number slot? Maybe. But then you'll have a bunch of stuff expecting version numbers that will stop working. > If I support "Metadata" : "false" in GemStone, I do not intend to > fabricate a "realistic looking Monticello version number" ... but I will > look into using the short SHA (when in a git repo) and perhaps fall back > to cypress.1 for non-git repos... > > Anyway, I will now be able to move forward with my Metacello Cypress > experiments and also try to understand how Metacello loads are affected > by metadtaless, since you _are_ fabricating Monticello version numbers, > my previous assumptions are not correct ... Please tell how it goes, especially the part with the short SHA, because I haven't tried that; I kept creating version numbers to just have gitfiletree behaving like filetree (apparently). Thierry > Thanks again! > > Dale > > On 6/28/16 4:11 PM, Dale Henrichs wrote: >> >> >> On 6/28/16 2:16 PM, Thierry Goubier wrote: >>> Dale, >>> >>> I'm sure it is possible. Wait, wait! If you have in your .filetree >>> "Metadata" : "false" then this is fine and it has switched to the >>> metadata-less mode. >>> >>> To see the changes on disk, you need to save a new version of your >>> packages, that should be all. >>> >>> I just tried and that works. >>> >>> 1- remove the GitFileTree repository from your image >>> 2- write the property "Metadata" : "false" in the .filetree on disk >>> 3- re-add the GitFileTree repository (local) >>> 4- modify then save one of the repository packages >>> 5- look on disk: no more monticello.meta/version! >>> >>> Note that I had no packages in the image linked to that repository at >>> 1-, because I'm not sure the simple removal would have really removed >>> the repository singleton object. >> Well I've found the culprit: MCFileTreeWriter>>addString:at:encodedTo: >> ensures that monticello.meta directory exists... >> >> Of course as I reread your comment "no more monticello.meta/version" I >> will have to say that I've only been looking at whether or not the >> monticello.meta directory existed or not ... I equate "no monticello >> meta data" as no monticello.meta directory and you interpret it as "no >> monticallo version file" ... well I can live with that ... I swore >> that I saw version files being updated in some of my experiments ... >> but now that I know that I should only look at the >> monticello.meta/version file, I will try yet again ... >> >> Thanks for you patience and help ... >> >> Dale >> > > > |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Dale Henrichs-3
Le 29/06/2016 00:55, Dale Henrichs a écrit :
> ... >> > I'm pretty certain the MCLazyVersionInfo is the real culprit here ... > while reading the code I recognized that many of the basic patterns were > exactly as I had remembered them from years ago ... however ... > MCLazyVersionInfo this puppy with its "default behavior" to scan the > universe is the real culprit ... I would think that at a minimum the > repository or repository group would/could be know at the time that the > MCLazyVersionInfo was created and a scan of just those repositories --- > already associated with the project --- would not be nearly as bad as > when we have now ... The MCLazyVersionInfo thing is mine too; it was a solution to avoid keeping MBs of version info kept inside the image memory, with the cost of having to reload that information when you access the ancestry. Now, the approach needs to be tuned to avoid spurious "query the world" searches, but, as you point out, I hasn't been too successfull yet. And one of the thing MC lack, is that link between a repository and a working copy. (at the same time, restricting version number determination to a subset of the repositories is against MC principles when numbering versions). Thierry |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Dale Henrichs-3
Dale,
you're right, the monticello.meta directory is kept, in part because I wasn't sure I could get rid of some of the information in some files in that directory (dependencies, initializers). Since those files don't usually create conflicts with git, it seemed Ok. Thierry Le 29/06/2016 01:11, Dale Henrichs a écrit : > > > On 6/28/16 2:16 PM, Thierry Goubier wrote: >> Dale, >> >> I'm sure it is possible. Wait, wait! If you have in your .filetree >> "Metadata" : "false" then this is fine and it has switched to the >> metadata-less mode. >> >> To see the changes on disk, you need to save a new version of your >> packages, that should be all. >> >> I just tried and that works. >> >> 1- remove the GitFileTree repository from your image >> 2- write the property "Metadata" : "false" in the .filetree on disk >> 3- re-add the GitFileTree repository (local) >> 4- modify then save one of the repository packages >> 5- look on disk: no more monticello.meta/version! >> >> Note that I had no packages in the image linked to that repository at >> 1-, because I'm not sure the simple removal would have really removed >> the repository singleton object. > Well I've found the culprit: MCFileTreeWriter>>addString:at:encodedTo: > ensures that monticello.meta directory exists... > > Of course as I reread your comment "no more monticello.meta/version" I > will have to say that I've only been looking at whether or not the > monticello.meta directory existed or not ... I equate "no monticello > meta data" as no monticello.meta directory and you interpret it as "no > monticallo version file" ... well I can live with that ... I swore that > I saw version files being updated in some of my experiments ... but now > that I know that I should only look at the monticello.meta/version file, > I will try yet again ... Yes > Thanks for you patience and help ... > > Dale > > > |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Dale Henrichs-3
Dale,
Le 29/06/2016 00:55, Dale Henrichs a écrit : > > ... > Actually, Metacello has a registry of loaded projects (baselines and > configurations) and using Metacello it is possible to determine which > pacakges belong to which projects etc. so it is not necessary to > "register packages with a project" ... the project registry in Metacello > is enough ... How do one adds a new package to a project in that registry? One can create new packages in a repository, and add them to the baseline (by editing the baseline method)? Do we need to rethink the Kommitter/Versionner stuff to provide a GUI to manipulate such things? Thierry |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Thierry Goubier
Hi Dale, Therry, I am working on a libgit-based git integration for Pharo and I am facing the same problem about version numbers. So maybe I can help you if you are working on that. I had the same idea as Dale, using short SHAs instead of correlative numbers, but I didn't have the time yet to got down that path. Therry, you said that
Do you already know which stuff will stop working? That would be realy helpful. |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
Hi Nicolas,
Le 29/06/2016 10:24, Nicolas Passerini a écrit : > Hi Dale, Therry, I am working on a libgit-based git integration for > Pharo and I am facing the same problem about version numbers. So maybe I > can help you if you are working on that. How is the libgit integration going? > I had the same idea as Dale, using short SHAs instead of correlative > numbers, but I didn't have the time yet to got down that path. Ok. > Therry, you said that > > > you'll have a bunch of stuff expecting version numbers that will > stop working. > > Do you already know which stuff will stop working? That would be realy > helpful. > All the Monticello GUI : track down version numbers to order stuff in the lists views everywhere. The Metacello / Gofer stuff (Configurations, Baselines) also use version numbers. One of the difficulty of switching will be the cohabitation of number-based systems (smalltalkhub) and SHA-based systems, especially when you do things like copying a version from a git repository to a smalltalkhub repository. That's why I suggested to change the ordering relation to a partial order based on the property A is an ancestor of B -> B is before A. Thierry |
|
|
In reply to this post by Thierry Goubier
On Wed, Jun 29, 2016 at 4:00 PM, Thierry Goubier
<[hidden email]> wrote: > Le 29/06/2016 00:55, Dale Henrichs a écrit : >> >> > ... >>> >>> >> I'm pretty certain the MCLazyVersionInfo is the real culprit here ... >> while reading the code I recognized that many of the basic patterns were >> exactly as I had remembered them from years ago ... however ... >> MCLazyVersionInfo this puppy with its "default behavior" to scan the >> universe is the real culprit ... I would think that at a minimum the >> repository or repository group would/could be know at the time that the >> MCLazyVersionInfo was created and a scan of just those repositories --- >> already associated with the project --- would not be nearly as bad as >> when we have now ... > > > The MCLazyVersionInfo thing is mine too; it was a solution to avoid keeping > MBs of version info kept inside the image memory, with the cost of having to > reload that information when you access the ancestry. Would it be feasible to drop all those MB except keep the latest version of a package in-Image, i.e. the one required to determine the next version number of the package. ? cheers -ben >> Now, the approach needs to be tuned to avoid spurious "query the world" > searches, but, as you point out, I hasn't been too successfull yet. And one > of the thing MC lack, is that link between a repository and a working copy. > > (at the same time, restricting version number determination to a subset of > the repositories is against MC principles when numbering versions). > > Thierry > |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
Le 29/06/2016 12:57, Ben Coman a écrit :
> On Wed, Jun 29, 2016 at 4:00 PM, Thierry Goubier > <[hidden email]> wrote: >> Le 29/06/2016 00:55, Dale Henrichs a écrit : >>> >>> >> ... >>>> >>>> >>> I'm pretty certain the MCLazyVersionInfo is the real culprit here ... >>> while reading the code I recognized that many of the basic patterns were >>> exactly as I had remembered them from years ago ... however ... >>> MCLazyVersionInfo this puppy with its "default behavior" to scan the >>> universe is the real culprit ... I would think that at a minimum the >>> repository or repository group would/could be know at the time that the >>> MCLazyVersionInfo was created and a scan of just those repositories --- >>> already associated with the project --- would not be nearly as bad as >>> when we have now ... >> >> >> The MCLazyVersionInfo thing is mine too; it was a solution to avoid keeping >> MBs of version info kept inside the image memory, with the cost of having to >> reload that information when you access the ancestry. > > Would it be feasible to drop all those MB except keep the latest > version of a package in-Image, > i.e. the one required to determine the next version number of the package. ? This is not considered enough for Monticello. If you are on a multi-people team with a smalltalkhub repository, keeping memory of myPackage.7 is not enough because your team members may push .8, .9, etc... before your save, and your save should be .10, not .8. So an access to the remote repository is necessary (and all of them; maybe you're working with multiple smalltalkhub repositories for that package). Of course, MC does not lock the remote repository while you do that, so the number it generates at the beginning (just before you get the window where you fill in the package version comment) is not guaranteed to be up to date if a team member saves a new version in the mean-time. Of course, MC does not forbid you to change by hand the version number. Thierry > cheers -ben > >>> Now, the approach needs to be tuned to avoid spurious "query the world" >> searches, but, as you point out, I hasn't been too successfull yet. And one >> of the thing MC lack, is that link between a repository and a working copy. >> >> (at the same time, restricting version number determination to a subset of >> the repositories is against MC principles when numbering versions). >> >> Thierry >> > > |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Thierry Goubier
Thank you Thierry! I have more questions inline. On Wed, Jun 29, 2016 at 10:44 AM, Thierry Goubier <[hidden email]> wrote:
So, if we built a new GUI which does not rely on those version numbers maybe we could get rid of that problem, what do you think?
Yes, I understand that they use version numbers, but do they require that version numbers are correlative?
I think that currently it is not possible to copy a number-based package version from smalltalkhub to a metadata-less git repository using git file tree. I mean, you can but the new version will not have the same version number, right?
I've lost you there. I agree that it is true that A is an ancestor of B -> B is before A, but the most common problem goes the other way arround, i.e. you want to know if A is ancestor of B and the implication then is not true: B is before A 'does not imply' B is ancestor of A. So I missed your point here. |
Re: How do I convert a FileTree project to Metadataless GitFileTree project?
|
|
In reply to this post by Thierry Goubier
On 6/28/16 10:22 PM, Thierry Goubier wrote: > Hi Dale, > > Le 29/06/2016 01:50, Dale Henrichs a écrit : >> Thierry, >> >> Okay ... it is "working" now ... I was also misled by the fact that you >> are continuing to fabricate Monticello version numbers which presumably >> cannot be relied upon in any way. >> >> Tugrik-Help-DaleHenrichs.11 will show up in each branch that includes >> the commit for "Tugrik-Help-DaleHenrichs.10", but the SHA and contents >> would be different for each one of the "Tugrik-Help-DaleHenrichs.11" , >> is that right? > > Yes. > >> Perhaps using the short SHA in place of the "version number" would be >> safer and provide useful information in the version number slot? > > Maybe. But then you'll have a bunch of stuff expecting version numbers > that will stop working. around the universe... But at the end of the day, these are tools issues and we shouldn't let tools limitations stand in the way of progress ... With that said I certainly understand the issues ... The design of Metacello was almost completely driven by "tools issues" and of course FileTree was originally written to be completely compatible with the Monticello tools ... But by dropping monticello version data (actually making monticello version data optional) we are breaking the tools ... The fabricated version numbers that you are currently using do the job of not breaking the tools, but they break the semantics of Monticello and fool the developer in believing things that aren't true --- which is a bit dangerous in my opinion ... So at the end of the day, if we are really going to _support_ optional monticello version data, then we need to make a pass through the tools .... I built tODE from the ground up so that I could make the tools directly git aware ... > >> If I support "Metadata" : "false" in GemStone, I do not intend to >> fabricate a "realistic looking Monticello version number" ... but I will >> look into using the short SHA (when in a git repo) and perhaps fall back >> to cypress.1 for non-git repos... >> >> Anyway, I will now be able to move forward with my Metacello Cypress >> experiments and also try to understand how Metacello loads are affected >> by metadtaless, since you _are_ fabricating Monticello version numbers, >> my previous assumptions are not correct ... > > Please tell how it goes, especially the part with the short SHA, > because I haven't tried that; I kept creating version numbers to just > have gitfiletree behaving like filetree (apparently). Baselines and git, the monticello version number is not relevant any more ... as you point out above, the SHA will be problematic with regards to the handful of places in the system that parse package names to extract a version number and solving that problem by itself may not be practical -- it remains to be seen ... The other half of this equation is that new tools should really be built to work with git-based repositories ... a Metacello Project Browser as the "main focus" would relieve the pressure to be completely compatible with the Monticello Browser and open the way for the creation of a Cypress Browser for working with git/filetree repos... Dale |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |