The Trunk: SMBase-tpr.141.mcz

Nicolas Cellier uploaded a new version of SMBase to project The Trunk:
http://source.squeak.org/trunk/SMBase-tpr.141.mcz

==================== Summary ====================

Name: SMBase-tpr.141
Author: tpr
Time: 18 January 2019, 4:24:49.815196 pm
UUID: 4997d27a-e261-41e0-b019-7723c85a5c1b
Ancestors: SMBase-cmm.140

First step towards improving the SM loading stuff; allow up to 3 attempts to help with handling network slowness etc.
Next step should be using a webclient instead of the httpsocket and getting somewhat nice eror handling.

=============== Diff against SMBase-cmm.140 ===============

Item was changed:
  ----- Method: SMSqueakMap>>loadFullFrom: (in category 'private') -----
+ loadFullFrom: aServerName
+       "Contact the SqueakMap at aServerName, building the the url for this version and load a full map from scratch.
+       Allow several retries in case of net slowness etc."
+       | url zipped attempts mapContents |
+       url := 'http://' , aServerName , '/loadgz?mapversion=' , SMSqueakMap version , '&checkpoint=' , checkpointNumber asString.
+       attempts := 0.
- loadFullFrom: aServerName
-       "Contact the SqueakMap at the url <aSqueakMapUrl>
-       and load a full map from scratch."

+       [attempts := attempts + 1.
+       zipped := HTTPSocket httpGet: url.
+       zipped isString
+               ifTrue: ["awful legacy way to detect net error - use a proper technique
+                       when the httpsocket can be replaced with a webclient. For now, raise a generic exception"
+                       NetworkError signal]]
+               on: NetworkError
+               do: [:ex |
+                       attempts >= 3
+                               ifTrue: [self halt: 'Unable to load SqueakMap map update after ' , attempts asString , ' attempts'].
+                       ex retry].
+               
+       mapContents := zipped contents.
+       ((self checkVersion: mapContents)
+                       and: [mapContents ~= 'UPTODATE'])
+               ifTrue: [self saveCheckpoint: mapContents.
+                       self reload]!
-       | url  zipped |
-       url := 'http://', aServerName, '/loadgz?mapversion=', SMSqueakMap version, '&checkpoint=', checkpointNumber asString.
-       Transcript show: 'Fetch: ', (Time millisecondsToRun: [ zipped := (HTTPSocket httpGet: url) contents]) asString, ' ms';cr.
-       Transcript show: 'Size: ', zipped size asString, ' bytes';cr.
-       ((self checkVersion: zipped) and: [zipped ~= 'UPTODATE'])
-               ifTrue:[
-                       Transcript show: 'Save checkpoint to disk: ', (Time millisecondsToRun: [
-                       self saveCheckpoint: zipped]) asString, ' ms';cr.
-                       Transcript show: 'Full reload from checkpoint: ', (Time millisecondsToRun: [
-                       self reload]) asString, ' ms';cr.]!

Hi Nicolas, hi Tim, please understand _why_ there is slowness before trying to "fix" it.  The problem is not "network slowness" but server slowness.  You should've used the other tool available to you, the timeout setting, but you have to understand the dynamics of the server before you can do that, else this sort of rapid-retrying may actually make things slightly worse.

Thanks.

On Tue, Mar 3, 2020 at 1:19 PM <[hidden email]> wrote:

Nicolas Cellier uploaded a new version of SMBase to project The Trunk:
http://source.squeak.org/trunk/SMBase-tpr.141.mcz

==================== Summary ====================

Name: SMBase-tpr.141
Author: tpr
Time: 18 January 2019, 4:24:49.815196 pm
UUID: 4997d27a-e261-41e0-b019-7723c85a5c1b
Ancestors: SMBase-cmm.140

First step towards improving the SM loading stuff; allow up to 3 attempts to help with handling network slowness etc.
Next step should be using a webclient instead of the httpsocket and getting somewhat nice eror handling.

=============== Diff against SMBase-cmm.140 ===============

Item was changed:
  ----- Method: SMSqueakMap>>loadFullFrom: (in category 'private') -----
+ loadFullFrom: aServerName
+       "Contact the SqueakMap at aServerName, building the the url for this version and load a full map from scratch.
+       Allow several retries in case of net slowness etc."
+       | url zipped attempts mapContents |
+       url := 'http://' , aServerName , '/loadgz?mapversion=' , SMSqueakMap version , '&checkpoint=' , checkpointNumber asString.
+       attempts := 0.
- loadFullFrom: aServerName
-       "Contact the SqueakMap at the url <aSqueakMapUrl>
-       and load a full map from scratch."

+       [attempts := attempts + 1.
+       zipped := HTTPSocket httpGet: url.
+       zipped isString
+               ifTrue: ["awful legacy way to detect net error - use a proper technique
+                       when the httpsocket can be replaced with a webclient. For now, raise a generic exception"
+                       NetworkError signal]]
+               on: NetworkError
+               do: [:ex |
+                       attempts >= 3
+                               ifTrue: [self halt: 'Unable to load SqueakMap map update after ' , attempts asString , ' attempts'].
+                       ex retry].
+               
+       mapContents := zipped contents.
+       ((self checkVersion: mapContents)
+                       and: [mapContents ~= 'UPTODATE'])
+               ifTrue: [self saveCheckpoint: mapContents.
+                       self reload]!
-       | url  zipped |
-       url := 'http://', aServerName, '/loadgz?mapversion=', SMSqueakMap version, '&checkpoint=', checkpointNumber asString.
-       Transcript show: 'Fetch: ', (Time millisecondsToRun: [ zipped := (HTTPSocket httpGet: url) contents]) asString, ' ms';cr.
-       Transcript show: 'Size: ', zipped size asString, ' bytes';cr.
-       ((self checkVersion: zipped) and: [zipped ~= 'UPTODATE'])
-               ifTrue:[
-                       Transcript show: 'Save checkpoint to disk: ', (Time millisecondsToRun: [
-                       self saveCheckpoint: zipped]) asString, ' ms';cr.
-                       Transcript show: 'Full reload from checkpoint: ', (Time millisecondsToRun: [
-                       self reload]) asString, ' ms';cr.]!