.mcd to .mcz (urgent)
Locked
 
|
Hi All,
so *how* do I ensure I have a .mcz when there appears to only be a .mcd? I'm talking about trying to get 'Kernel-ul.875.mcz' from trunk when Kernel-ul.875(eem.872).mcd appears to be on offer. If I try MCCacheRepository default storeVersion: (trunk versionNamed: 'Kernel-ul.875.mcz') I *do not* get a 'Kernel-ul.875.mcz' in my cache repository. I'm f*&^%d until I can get this working. help, please! -- Eliot
|
|
|
OK here's what I came up with, and it's not pretty. But it is useful. Should this be included in MCCacheRepository or MCFileBasedRepository as a general facility? IMO, yes, to have some poor soul going down this same route. But I want to canvas opinion first. Nte that one wpould think all one had to do was cacheRepo storeVersion: (repo versionNamed: versionName). but MCCacheRepository defeats that simple route with MCCacheRepository >>basicStoreVersion: aVersion (aVersion isCacheable not or: [self allFileNames includes: aVersion fileName]) ifFalse: [super basicStoreVersion: aVersion] So that leaves the brute-force: cachedNonDiffyVersionNamed: versionName from: repo "Make sure that the cache contains a non-diffy version of versionName and answer it." | cacheRepo | self assert: (versionName endsWith: '.mcz'). cacheRepo := MCCacheRepository default. "Make sure that at least the diffy (.mcd) version is present" (cacheRepo directory includesKey: versionName) ifFalse: [cacheRepo storeVersion: (repo versionNamed: versionName)]. "if after storeVersion there's still no .mcz we need to create one; sigh..." (cacheRepo directory includesKey: versionName) ifFalse: [| baseName diffyVersionName diffyVersion file delete | baseName := versionName allButLast: 4. "strip .mcz" diffyVersionName := cacheRepo directory fileNames detect: [:fn| (fn endsWith: '.mcd') and: [(fn copyUpTo: $() = baseName]]. diffyVersion := cacheRepo versionNamed: diffyVersionName. file := cacheRepo directory newFileNamed: versionName. delete := false. [file binary. [MCMczWriter fileOut: diffyVersion on: file] on: Error do: [:ex| delete := true. "don't leave half-formed .mcz files around to screw things up later on..." ex pass]] ensure: [file close. delete ifTrue: [cacheRepo directory deleteFileNamed: versionName]]. "now delete the damn diffy version that caused all the pain in the first place" delete ifFalse: [cacheRepo directory deleteFileNamed: diffyVersionName]. cacheRepo cacheAllFilenames]. ^cacheRepo versionNamed: versionName On Wed, Oct 8, 2014 at 4:12 PM, Eliot Miranda <[hidden email]> wrote:
best, Eliot
|
|
|
Hi, I'm sure it would help to understand the broader context of why
the .mcd doesn't meet your needs. I assume you must have some external (non-Smalltalk) tool unzipping the mcz to get at the contents? Because if you were only accessing via the MCRepository API's then mcd shouldn't matter. I think you're on the right track with the cacheRepo storeVersion: (repo versionNamed: versionName). approach, because that lets the Repository "do its job" the way it wants or needs to, able to assume its users would only care about the objects returned, not its private, internal storage format. There is a 'alwaysStoreDiffs' option for every Repository, so it might help to make sure that's turned off on your CacheRepository. But I know that's not what you want if the incoming is _already_ a mcd, because you want a 'alwaysStoreFull', but I'm not sure why.. On Wed, Oct 8, 2014 at 7:37 PM, Eliot Miranda <[hidden email]> wrote: > OK here's what I came up with, and it's not pretty. But it is useful. > Should this be included in MCCacheRepository or MCFileBasedRepository as a > general facility? IMO, yes, to have some poor soul going down this same > route. But I want to canvas opinion first. Nte that one wpould think all > one had to do was > > cacheRepo storeVersion: (repo versionNamed: versionName). > > but MCCacheRepository defeats that simple route with > > MCCacheRepository >>basicStoreVersion: aVersion > (aVersion isCacheable not or: [self allFileNames includes: aVersion > fileName]) > ifFalse: [super basicStoreVersion: aVersion] > > So that leaves the brute-force: > > cachedNonDiffyVersionNamed: versionName from: repo > "Make sure that the cache contains a non-diffy version of versionName and > answer it." > | cacheRepo | > self assert: (versionName endsWith: '.mcz'). > cacheRepo := MCCacheRepository default. > "Make sure that at least the diffy (.mcd) version is present" > (cacheRepo directory includesKey: versionName) ifFalse: > [cacheRepo storeVersion: (repo versionNamed: versionName)]. > "if after storeVersion there's still no .mcz we need to create one; sigh..." > (cacheRepo directory includesKey: versionName) ifFalse: > [| baseName diffyVersionName diffyVersion file delete | > baseName := versionName allButLast: 4. "strip .mcz" > diffyVersionName := cacheRepo directory fileNames detect: [:fn| (fn > endsWith: '.mcd') and: [(fn copyUpTo: $() = baseName]]. > diffyVersion := cacheRepo versionNamed: diffyVersionName. > file := cacheRepo directory newFileNamed: versionName. > delete := false. > [file binary. > [MCMczWriter fileOut: diffyVersion on: file] > on: Error > do: [:ex| > delete := true. "don't leave half-formed .mcz files around to screw things > up later on..." > ex pass]] > ensure: > [file close. > delete ifTrue: > [cacheRepo directory deleteFileNamed: versionName]]. > "now delete the damn diffy version that caused all the pain in the first > place" > delete ifFalse: > [cacheRepo directory deleteFileNamed: diffyVersionName]. > cacheRepo cacheAllFilenames]. > ^cacheRepo versionNamed: versionName > > On Wed, Oct 8, 2014 at 4:12 PM, Eliot Miranda <[hidden email]> > wrote: >> >> Hi All, >> >> so *how* do I ensure I have a .mcz when there appears to only be a >> .mcd? I'm talking about trying to get 'Kernel-ul.875.mcz' from trunk when >> Kernel-ul.875(eem.872).mcd appears to be on offer. If I try >> >> MCCacheRepository default storeVersion: (trunk versionNamed: >> 'Kernel-ul.875.mcz') I *do not* get a 'Kernel-ul.875.mcz' in my cache >> repository. I'm f*&^%d until I can get this working. help, please! >> >> -- >> best, >> Eliot > > > > > -- > best, > Eliot > > > |
|
|
On Wed, Oct 8, 2014 at 5:59 PM, Chris Muller <[hidden email]> wrote: Hi, I'm sure it would help to understand the broader context of why The Spur bootstrap depends on being able to generate patched files. I could, had I known the syetem better, have produced those patches as mcd's, but have produced them as .mczs. IMO mcd is a premature optimization, saving a littel bit I think you're on the right track with the There really should be an "If I say store a version I mean it" API that means that diffy versions can be stored. It's a PITA not having this.
Um, the cache repository's value is indeed false (actually nil, which defaults to false). But the storeVersion code doesn't pay attention to that flag.
best, Eliot
|
|
|
On Wed, Oct 8, 2014 at 6:05 PM, Eliot Miranda <[hidden email]> wrote:
oops. ignore the partial sentence. I had meant to delete it ;-)
best, Eliot
|
|
|
In reply to this post by Eliot Miranda-2
On Wed, Oct 8, 2014 at 8:05 PM, Eliot Miranda <[hidden email]> wrote:
> > > On Wed, Oct 8, 2014 at 5:59 PM, Chris Muller <[hidden email]> wrote: >> >> Hi, I'm sure it would help to understand the broader context of why >> the .mcd doesn't meet your needs. I assume you must have some >> external (non-Smalltalk) tool unzipping the mcz to get at the >> contents? Because if you were only accessing via the MCRepository >> API's then mcd shouldn't matter. > > > The Spur bootstrap depends on being able to generate patched files. I > could, had I known the syetem better, have produced those patches as mcd's, > but have produced them as .mczs. Okay, but when you generate those patched mcd's, they are still based off the prior version, so I'm still having trouble understanding why Spur bootstrap would have any trouble reading a mcd.. >IMO mcd is a premature optimization, > saving a littel bit > >> I think you're on the right track with the >> >> cacheRepo storeVersion: (repo versionNamed: versionName). >> >> approach, because that lets the Repository "do its job" the way it >> wants or needs to, able to assume its users would only care about the >> objects returned, not its private, internal storage format. > > There really should be an "If I say store a version I mean it" API that > means that diffy versions can be stored. It's a PITA not having this. It's an internal format that's supposed to be invisible to users of Repository's. I think exposing it could create PITA's.. >> There is a 'alwaysStoreDiffs' option for every Repository, so it might >> help to make sure that's turned off on your CacheRepository. But I >> know that's not what you want if the incoming is _already_ a mcd, >> because you want a 'alwaysStoreFull', but I'm not sure why.. > > > Um, the cache repository's value is indeed false (actually nil, which > defaults to false). But the storeVersion code doesn't pay attention to that > flag. Yes it does, it calls #prepareVersionForStorage: which checks it. As I said, it won't convert a mcd to a mcz, it will only convert a mcz to a mcd, which Spur should be able to read just fine assuming the ancestor of the patched package is available in a repository...? >> On Wed, Oct 8, 2014 at 7:37 PM, Eliot Miranda <[hidden email]> >> wrote: >> > OK here's what I came up with, and it's not pretty. But it is useful. >> > Should this be included in MCCacheRepository or MCFileBasedRepository as >> > a >> > general facility? IMO, yes, to have some poor soul going down this same >> > route. But I want to canvas opinion first. Nte that one wpould think >> > all >> > one had to do was >> > >> > cacheRepo storeVersion: (repo versionNamed: versionName). >> > >> > but MCCacheRepository defeats that simple route with >> > >> > MCCacheRepository >>basicStoreVersion: aVersion >> > (aVersion isCacheable not or: [self allFileNames includes: aVersion >> > fileName]) >> > ifFalse: [super basicStoreVersion: aVersion] >> > >> > So that leaves the brute-force: >> > >> > cachedNonDiffyVersionNamed: versionName from: repo >> > "Make sure that the cache contains a non-diffy version of versionName >> > and >> > answer it." >> > | cacheRepo | >> > self assert: (versionName endsWith: '.mcz'). >> > cacheRepo := MCCacheRepository default. >> > "Make sure that at least the diffy (.mcd) version is present" >> > (cacheRepo directory includesKey: versionName) ifFalse: >> > [cacheRepo storeVersion: (repo versionNamed: versionName)]. >> > "if after storeVersion there's still no .mcz we need to create one; >> > sigh..." >> > (cacheRepo directory includesKey: versionName) ifFalse: >> > [| baseName diffyVersionName diffyVersion file delete | >> > baseName := versionName allButLast: 4. "strip .mcz" >> > diffyVersionName := cacheRepo directory fileNames detect: [:fn| (fn >> > endsWith: '.mcd') and: [(fn copyUpTo: $() = baseName]]. >> > diffyVersion := cacheRepo versionNamed: diffyVersionName. >> > file := cacheRepo directory newFileNamed: versionName. >> > delete := false. >> > [file binary. >> > [MCMczWriter fileOut: diffyVersion on: file] >> > on: Error >> > do: [:ex| >> > delete := true. "don't leave half-formed .mcz files around to screw >> > things >> > up later on..." >> > ex pass]] >> > ensure: >> > [file close. >> > delete ifTrue: >> > [cacheRepo directory deleteFileNamed: versionName]]. >> > "now delete the damn diffy version that caused all the pain in the first >> > place" >> > delete ifFalse: >> > [cacheRepo directory deleteFileNamed: diffyVersionName]. >> > cacheRepo cacheAllFilenames]. >> > ^cacheRepo versionNamed: versionName >> > >> > On Wed, Oct 8, 2014 at 4:12 PM, Eliot Miranda <[hidden email]> >> > wrote: >> >> >> >> Hi All, >> >> >> >> so *how* do I ensure I have a .mcz when there appears to only be a >> >> .mcd? I'm talking about trying to get 'Kernel-ul.875.mcz' from trunk >> >> when >> >> Kernel-ul.875(eem.872).mcd appears to be on offer. If I try >> >> >> >> MCCacheRepository default storeVersion: (trunk versionNamed: >> >> 'Kernel-ul.875.mcz') I *do not* get a 'Kernel-ul.875.mcz' in my cache >> >> repository. I'm f*&^%d until I can get this working. help, please! >> >> >> >> -- >> >> best, >> >> Eliot >> > >> > >> > >> > >> > -- >> > best, >> > Eliot >> > >> > >> > >> > > > > -- > best, > Eliot |
|
|
On Wed, Oct 8, 2014 at 6:20 PM, Chris Muller <[hidden email]> wrote: On Wed, Oct 8, 2014 at 8:05 PM, Eliot Miranda <[hidden email]> wrote: I don't generate patches mcds. I generate patched mczs. The issue is that some versions are only available as mcds and need to be cnverted into mczs before they can be patched. An annoying associated issue is that the query available version interface blurs the distinction between mczs and mcds so if you ask for an mcz and the repository has an mcd it'll tell you it has the version but it doesn't. It only has the mcd. Further there's no convenient way to get the mcz form the mcd. Thats why I had to write the below, that constructs the full mcz form the mcd.
mcd is *not* an internal format. If one interacts with repositories and asks for mczs one gets given mcds if these are what's available. It would be *great* if they were an internal format but I assure you they're not.
Did you see my comment below re basicStoreVersion: ? Prepare version for storage will arrange to store a diffy version from a full version, but it *won't* arrange to store a full version from a diffy version. But later on MCCacheRepository >>basicStoreVersion: refuses to store anything anyway. I wouldn't have written the below if mcd's weren't a problem. I'm not a masochist ;-)
best, Eliot
|
|
|
On Wed, Oct 8, 2014 at 9:54 PM, Eliot Miranda <[hidden email]> wrote:
> > > On Wed, Oct 8, 2014 at 6:20 PM, Chris Muller <[hidden email]> wrote: >> >> On Wed, Oct 8, 2014 at 8:05 PM, Eliot Miranda <[hidden email]> >> wrote: >> > >> > >> > On Wed, Oct 8, 2014 at 5:59 PM, Chris Muller <[hidden email]> >> > wrote: >> >> >> >> Hi, I'm sure it would help to understand the broader context of why >> >> the .mcd doesn't meet your needs. I assume you must have some >> >> external (non-Smalltalk) tool unzipping the mcz to get at the >> >> contents? Because if you were only accessing via the MCRepository >> >> API's then mcd shouldn't matter. >> > >> > >> > The Spur bootstrap depends on being able to generate patched files. I >> > could, had I known the syetem better, have produced those patches as >> > mcd's, >> > but have produced them as .mczs. >> >> Okay, but when you generate those patched mcd's, they are still based >> off the prior version, so I'm still having trouble understanding why >> Spur bootstrap would have any trouble reading a mcd.. > > > I don't generate patches mcds. I generate patched mczs. The issue is that > some versions are only available as mcds and need to be cnverted into mczs > before they can be patched. If there's a case where a distinction needs to be made between diffy's and full's when interacting with a Repository (other than performance tuning) then I'm terribly interested in understanding what it is, but you keep not telling me. > An annoying associated issue is that the query > available version interface blurs the distinction between mczs and mcds so > if you ask for an mcz and the repository has an mcd it'll tell you it has > the version but it doesn't. It only has the mcd. Further there's no > convenient way to get the mcz form the mcd. Thats why I had to write the > below, that constructs the full mcz form the mcd. > > >> >> >> >IMO mcd is a premature optimization, >> > saving a littel bit >> > >> >> I think you're on the right track with the >> >> >> >> cacheRepo storeVersion: (repo versionNamed: versionName). >> >> >> >> approach, because that lets the Repository "do its job" the way it >> >> wants or needs to, able to assume its users would only care about the >> >> objects returned, not its private, internal storage format. >> > >> > There really should be an "If I say store a version I mean it" API that >> > means that diffy versions can be stored. It's a PITA not having this. >> >> It's an internal format that's supposed to be invisible to users of >> Repository's. I think exposing it could create PITA's.. > > > mcd is *not* an internal format. If one interacts with repositories and > asks for mczs one gets given mcds if these are what's available. It would > be *great* if they were an internal format but I assure you they're not. Please forgive my curiousity not being quenched by mere "assurance". :) For your patching, I assume you need access to the MCDefinitions inside the Snapshot. Whether an mcd or mcz, when accessing through the MCRepository API, you've got them all (hence my wondering if it was for external tool access) as long as the ancestor be available. >> >> There is a 'alwaysStoreDiffs' option for every Repository, so it might >> >> help to make sure that's turned off on your CacheRepository. But I >> >> know that's not what you want if the incoming is _already_ a mcd, >> >> because you want a 'alwaysStoreFull', but I'm not sure why.. >> > >> > >> > Um, the cache repository's value is indeed false (actually nil, which >> > defaults to false). But the storeVersion code doesn't pay attention to >> > that >> > flag. >> >> Yes it does, it calls #prepareVersionForStorage: which checks it. As >> I said, it won't convert a mcd to a mcz, it will only convert a mcz to >> a mcd, which Spur should be able to read just fine assuming the >> ancestor of the patched package is available in a repository...? > > > Did you see my comment below re basicStoreVersion: ? Prepare version for > storage will arrange to store a diffy version from a full version, but it > *won't* arrange to store a full version from a diffy version. But later on > MCCacheRepository >>basicStoreVersion: refuses to store anything anyway. I > wouldn't have written the below if mcd's weren't a problem. I'm not a > masochist ;-) I don't see it. 'cache' in #basicStoreVersion does not refer to the cache repository... Anyway, if that works for you I couldn't do better without understanding why I was doing it. > > >> >> On Wed, Oct 8, 2014 at 7:37 PM, Eliot Miranda <[hidden email]> >> >> wrote: >> >> > OK here's what I came up with, and it's not pretty. But it is >> >> > useful. >> >> > Should this be included in MCCacheRepository or MCFileBasedRepository >> >> > as >> >> > a >> >> > general facility? IMO, yes, to have some poor soul going down this >> >> > same >> >> > route. But I want to canvas opinion first. Nte that one wpould >> >> > think >> >> > all >> >> > one had to do was >> >> > >> >> > cacheRepo storeVersion: (repo versionNamed: versionName). >> >> > >> >> > but MCCacheRepository defeats that simple route with >> >> > >> >> > MCCacheRepository >>basicStoreVersion: aVersion >> >> > (aVersion isCacheable not or: [self allFileNames includes: aVersion >> >> > fileName]) >> >> > ifFalse: [super basicStoreVersion: aVersion] >> >> > >> >> > So that leaves the brute-force: >> >> > >> >> > cachedNonDiffyVersionNamed: versionName from: repo >> >> > "Make sure that the cache contains a non-diffy version of versionName >> >> > and >> >> > answer it." >> >> > | cacheRepo | >> >> > self assert: (versionName endsWith: '.mcz'). >> >> > cacheRepo := MCCacheRepository default. >> >> > "Make sure that at least the diffy (.mcd) version is present" >> >> > (cacheRepo directory includesKey: versionName) ifFalse: >> >> > [cacheRepo storeVersion: (repo versionNamed: versionName)]. >> >> > "if after storeVersion there's still no .mcz we need to create one; >> >> > sigh..." >> >> > (cacheRepo directory includesKey: versionName) ifFalse: >> >> > [| baseName diffyVersionName diffyVersion file delete | >> >> > baseName := versionName allButLast: 4. "strip .mcz" >> >> > diffyVersionName := cacheRepo directory fileNames detect: [:fn| (fn >> >> > endsWith: '.mcd') and: [(fn copyUpTo: $() = baseName]]. >> >> > diffyVersion := cacheRepo versionNamed: diffyVersionName. >> >> > file := cacheRepo directory newFileNamed: versionName. >> >> > delete := false. >> >> > [file binary. >> >> > [MCMczWriter fileOut: diffyVersion on: file] >> >> > on: Error >> >> > do: [:ex| >> >> > delete := true. "don't leave half-formed .mcz files around to screw >> >> > things >> >> > up later on..." >> >> > ex pass]] >> >> > ensure: >> >> > [file close. >> >> > delete ifTrue: >> >> > [cacheRepo directory deleteFileNamed: versionName]]. >> >> > "now delete the damn diffy version that caused all the pain in the >> >> > first >> >> > place" >> >> > delete ifFalse: >> >> > [cacheRepo directory deleteFileNamed: diffyVersionName]. >> >> > cacheRepo cacheAllFilenames]. >> >> > ^cacheRepo versionNamed: versionName >> >> > >> >> > On Wed, Oct 8, 2014 at 4:12 PM, Eliot Miranda >> >> > <[hidden email]> >> >> > wrote: >> >> >> >> >> >> Hi All, >> >> >> >> >> >> so *how* do I ensure I have a .mcz when there appears to only be >> >> >> a >> >> >> .mcd? I'm talking about trying to get 'Kernel-ul.875.mcz' from >> >> >> trunk >> >> >> when >> >> >> Kernel-ul.875(eem.872).mcd appears to be on offer. If I try >> >> >> >> >> >> MCCacheRepository default storeVersion: (trunk versionNamed: >> >> >> 'Kernel-ul.875.mcz') I *do not* get a 'Kernel-ul.875.mcz' in my >> >> >> cache >> >> >> repository. I'm f*&^%d until I can get this working. help, please! >> >> >> >> >> >> -- >> >> >> best, >> >> >> Eliot >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > best, >> >> > Eliot >> >> > >> >> > >> >> > >> >> >> > >> > >> > >> > -- >> > best, >> > Eliot > > > > > -- > best, > Eliot |
|
|
In reply to this post by Eliot Miranda-2
On 09.10.2014, at 02:37, Eliot Miranda <[hidden email]> wrote:
> OK here's what I came up with, and it's not pretty. But it is useful. Should this be included in MCCacheRepository or MCFileBasedRepository as a general facility? IMO, yes, to have some poor soul going down this same route. But I want to canvas opinion first. Nte that one wpould think all one had to do was > > cacheRepo storeVersion: (repo versionNamed: versionName). The whole point of MCDiffyVersion is to be efficient, and avoiding having to create and store a full snapshot. The easiest way to achieve what you want would be something like cacheRepo storeVersion: (repo versionNamed: versionName) asNonDiffyVersion. MCVersion>>asNonDiffyVersion ^self MCDiffyVersion>>asNonDiffyVersion ^MCVersion package: package info: info snapshot: self snapshot dependencies: dependencies This is untested but sounds trivial enough that it might have the chance of being right ;) - Bert - |
smime.p7s (5K)
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |