faster process to move to gemstone

> yesterday I did my daily batch conversion to gemstone and I came up with a slightly faster process.
> If you use gofer I fear that all the monticello versions are completely parsed before being saved / uploaded.
>
> I tried to avoid this using wget to download all the versions and then script the upload
>
> ==========================================================================
> # bash, takes around 1min vs 10mins with gofer
> wget --mirror --page-requisites http://www.squeaksource.com/Citezen/
>
> # pharo
> f := FSFilesystem disk workingDirectory / 'www.squeaksource.com' / 'Citezen'.
> files := f glob: [ :f| f basename endsWith: '.mcz'].
>
> files do: [ :file ||entity stream|
> stream := file readStream.
> (entity := ZnStreamingEntity type: ZnMimeType applicationOctetStream)
> stream: stream;
> contentLength: stream size.
> Transcript show: file; cr.
> "upload the version to gemstone"
> ZnClient new
> systemPolicy;
> beOneShot;
> signalProgress: true;
> ifFail: [ :exception | Transcript show: file basename; show: ' '; print: exception ];
> username: 'XXX' password: 'XXX';
> entity: entity;
> url: ( 'http://ss3.gemstone.com/ss/citezen/', file basename);
> put ]
> displayingProgress: 'Uploading mzcs'
> ==========================================================================
>
> best
> cami

> Hey Camillo,
>
> On 15 Feb 2012, at 13:02, Camillo Bruni wrote:
>
>> yesterday I did my daily batch conversion to gemstone and I came up with a slightly faster process.
>> If you use gofer I fear that all the monticello versions are completely parsed before being saved / uploaded.
>>
>> I tried to avoid this using wget to download all the versions and then script the upload
>>
>> ==========================================================================
>> # bash, takes around 1min vs 10mins with gofer
>> wget --mirror --page-requisites http://www.squeaksource.com/Citezen/
>>
>> # pharo
>> f := FSFilesystem disk workingDirectory / 'www.squeaksource.com' / 'Citezen'.
>> files := f glob: [ :f| f basename endsWith: '.mcz'].
>>
>> files do: [ :file ||entity stream|
>> stream := file readStream.
>> (entity := ZnStreamingEntity type: ZnMimeType applicationOctetStream)
>> stream: stream;
>> contentLength: stream size.
>> Transcript show: file; cr.
>> "upload the version to gemstone"
>> ZnClient new
>> systemPolicy;
>> beOneShot;
>> signalProgress: true;
>> ifFail: [ :exception | Transcript show: file basename; show: ' '; print: exception ];
>> username: 'XXX' password: 'XXX';
>> entity: entity;
>> url: ( 'http://ss3.gemstone.com/ss/citezen/', file basename);
>> put ]
>> displayingProgress: 'Uploading mzcs'
>> ==========================================================================
>>
>> best
>> cami
>
> Cool stuff, beautiful script !
>
> I don't know enough about the internals of Gofer and/or Monticello, but doing what the wget does is easy in Smalltalk.
>
> repositoryUrl := 'http://www.squeaksource.com/ZincHTTPComponents/'.
> files := (MCHttpRepository new
> parseFileNamesFromStream: (ZnClient new get: repositoryUrl) entity readStream)
> select: [ :each | ZnMonticelloServerDelegate new isValidMczName: each ].
> files do: [ :each | | mcz |
> Transcript cr; show: each.
> mcz := (ZnClient new get: 'http://www.squeaksource.com/ZincHTTPComponents/', each) entity.
> "…" ].
>
> You can use the mcz entity directly as an argument to the next put.
>
> What do you think ?

> Hey Camillo,
>
> On 15 Feb 2012, at 13:02, Camillo Bruni wrote:
>
>> yesterday I did my daily batch conversion to gemstone and I came up with a slightly faster process.
>> If you use gofer I fear that all the monticello versions are completely parsed before being saved / uploaded.
>>
>> I tried to avoid this using wget to download all the versions and then script the upload
>>
>> ==========================================================================
>> # bash, takes around 1min vs 10mins with gofer
>> wget --mirror --page-requisites http://www.squeaksource.com/Citezen/
>>
>> # pharo
>> f := FSFilesystem disk workingDirectory / 'www.squeaksource.com' / 'Citezen'.
>> files := f glob: [ :f| f basename endsWith: '.mcz'].
>>
>> files do: [ :file ||entity stream|
>> stream := file readStream.
>> (entity := ZnStreamingEntity type: ZnMimeType applicationOctetStream)
>> stream: stream;
>> contentLength: stream size.
>> Transcript show: file; cr.
>> "upload the version to gemstone"
>> ZnClient new
>> systemPolicy;
>> beOneShot;
>> signalProgress: true;
>> ifFail: [ :exception | Transcript show: file basename; show: ' '; print: exception ];
>> username: 'XXX' password: 'XXX';
>> entity: entity;
>> url: ( 'http://ss3.gemstone.com/ss/citezen/', file basename);
>> put ]
>> displayingProgress: 'Uploading mzcs'
>> ==========================================================================
>>
>> best
>> cami
>
> Cool stuff, beautiful script !
>
> I don't know enough about the internals of Gofer and/or Monticello, but doing what the wget does is easy in Smalltalk.
>
> repositoryUrl := 'http://www.squeaksource.com/ZincHTTPComponents/'.
> files := (MCHttpRepository new
> parseFileNamesFromStream: (ZnClient new get: repositoryUrl) entity readStream)
> select: [ :each | ZnMonticelloServerDelegate new isValidMczName: each ].
> files do: [ :each | | mcz |
> Transcript cr; show: each.
> mcz := (ZnClient new get: 'http://www.squeaksource.com/ZincHTTPComponents/', each) entity.
> "…" ].
>
> You can use the mcz entity directly as an argument to the next put.
>
> What do you think ?
>
> Let me know how fast is goes.
>
> Sven
>
>
>
>
>
>